-- Logs begin at Wed 2022-08-31 09:00:50 UTC, end at Wed 2022-08-31 10:07:43 UTC. -- Aug 31 09:33:23.896704 dv-cinder-837543-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 09:33:23.905083 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 09:33:23 2022] *** Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: nodename: dv-cinder-837543-8-iscsi Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: machine: x86_64 Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: clock source: unix Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: pcre jit disabled Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: detected number of CPU cores: 4 Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: current working directory: / Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: detected binary path: /usr/bin/uwsgi-core Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: your processes number limit is 47680 Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: your memory page size is 4096 bytes Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: detected max file descriptor number: 2048 Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: lock engine: pthread robust mutexes Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: thunder lock: enabled Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 09:33:23.909942 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 09:33:23.955865 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: Python main interpreter initialized at 0x55742edec9c0 Aug 31 09:33:23.955865 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: python threads support enabled Aug 31 09:33:23.955865 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: your server socket listen backlog is limited to 100 connections Aug 31 09:33:23.955865 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: your mercy for graceful operations on workers is 90 seconds Aug 31 09:33:23.956369 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 09:33:23.956369 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: *** Operational MODE: preforking *** Aug 31 09:33:23.956623 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: *** uWSGI is running in multiple interpreter mode *** Aug 31 09:33:23.956623 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: spawned uWSGI master process (pid: 93516) Aug 31 09:33:23.956810 dv-cinder-837543-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 09:33:23.958503 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: spawned uWSGI worker 1 (pid: 93532, cores: 1) Aug 31 09:33:23.959721 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: spawned uWSGI worker 2 (pid: 93533, cores: 1) Aug 31 09:33:23.959721 dv-cinder-837543-8-iscsi devstack@keystone.service[93516]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 09:33:28.376462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 09:33:28.376462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: from cryptography.utils import int_from_bytes Aug 31 09:33:28.377554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 09:33:28.377554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: from cryptography.utils import int_from_bytes Aug 31 09:33:28.428593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 09:33:28.428852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 09:33:28.428983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] command line args: [] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 09:33:28.429194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 09:33:28.429368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ================================================================================ {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 09:33:28.429621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] admin_token = **** {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.429860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] config_dir = [] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.430045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.430222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] config_source = [] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.430396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] debug = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.430592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.430998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.431160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.431403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.431646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.431868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] list_limit = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.432044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] log_config_append = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.432222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.432402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] log_dir = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.432575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] log_file = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.432745 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.432974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.433212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.433452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.433632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.433795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.434014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.434363 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.434598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.434829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.435061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.435467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.435650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.435840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.436017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.436523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.436733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] publish_errors = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.436933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.437119 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.437300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.437513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.437743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.437974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] standard_threads = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.438203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.438442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.438640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.438825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] use_journal = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.439009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] use_json = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.439219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] use_stderr = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.439416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] use_syslog = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.439609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.439803 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.439989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.440200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.440438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.440681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.440917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.441162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.441395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.441639 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.441882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.442129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.442359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.442605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.442836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.443077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.443348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.443550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.443806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.444054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.444298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.444531 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.444781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.445038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.445287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.445528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.445761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.445987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.446214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.446449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.446697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.446941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.447190 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.447421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.447665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.447889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.448125 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.448369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.448597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.448830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.449063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.449289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.449513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.449769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.450010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.450278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.450462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.450691 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.450924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.451166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.451411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.451646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.451894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.452122 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.452354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.452586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.452817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.453048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.453280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.453506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.453750 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.453983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.454216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.454444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.454701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.454934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.455167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.455414 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.455652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.455924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.456158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.456401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.456644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.456879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.457111 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.457345 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.457589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.457819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.458153 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.458367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.458546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.458742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.458920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.459176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.459413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.459645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.459876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.460168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.460365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.460588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.460826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.461057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.461296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.461527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.461760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.461990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.462249 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.462439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.462658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.462893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.463163 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.463405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.463607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.463833 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.464067 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.464318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.464575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.464813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.465042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] auth.external = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.465272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.466113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.466328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.466558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] auth.password = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.466784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] auth.token = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.467023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.467261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.467515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.467725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.467975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.468218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.468447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.468680 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] credential.caching = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.468925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.469167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.469412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.469651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.469891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.470123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.470366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.470616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.470851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.471100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.471413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.471655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.471952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.472139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.472387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.472622 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] federation.caching = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.472857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.473101 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.473342 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.473569 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.473816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.474051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.474299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.474531 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.474786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.475020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.475289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.475524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.caching = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.475768 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.476021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.476254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.476487 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.476730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.476958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.477197 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.477443 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.477692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.477919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.478150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.478377 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.478621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.478863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.479106 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.479410 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.479653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.479856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.480089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.480322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.480550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.480783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.481367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.481367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.481713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.481899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.482104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.482292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.482458 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.482770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.482989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.483207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.483415 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.483612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.483810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.484039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.484274 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.484507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.484752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.484988 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.485223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.485472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.485713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.486176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.486421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.486664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.486901 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.487195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.487388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.487607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.487826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.488090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.488317 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.488560 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.488802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.489046 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.489280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.489508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.489740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.489973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.490205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.490433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.490676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.490917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.491180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.491425 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.491668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.491895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.492140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.492373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.492608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.492840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.493071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.493312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.493553 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.493789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.494041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.494268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.494511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.494744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.494978 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.495235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.495622 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.495866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.496077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.496323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.496565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] resource.caching = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.497347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.497594 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.497829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.498076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.498324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.498560 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.498805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.499041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.499304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.499540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] role.caching = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.499786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] role.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.500013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.500260 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.500506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.500749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.500993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.501234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.501477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.501718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.501960 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.502192 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.502435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.502679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.502920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.503189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.503602 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.503835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.504078 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.504324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.504568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.504815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.505051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.505287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.505537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.505776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.506003 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.506239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.506469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.506717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.506968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.507237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.507476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.507710 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.507940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.508176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] token.caching = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.508412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.508658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.508892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.509145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.509394 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.509625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.509866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.510108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.510351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.510583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.510819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.511054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.511319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.511596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.511822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.512065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.512264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 09:33:28.512264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: from cryptography.utils import int_from_bytes Aug 31 09:33:28.513293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.513481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 09:33:28.513481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: from cryptography.utils import int_from_bytes Aug 31 09:33:28.513829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.connection = **** {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.513829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.513829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.513829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.513829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.513829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.514847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.514847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.514847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.514847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.515900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.515900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.515900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.515900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.516586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.516586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.516586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.517118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.517118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.517432 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 09:33:28.549812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.550098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.550401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.559600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.562631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 09:33:28.562806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 09:33:28.563006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] command line args: [] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 09:33:28.563299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 09:33:28.563488 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ================================================================================ {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 09:33:28.563716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] admin_token = **** {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.563962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] config_dir = [] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.564148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.564330 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] config_source = [] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.564505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] debug = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.564749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.564940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.565120 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.565387 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.565634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.565865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] list_limit = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.566042 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] log_config_append = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.566239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.566370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] log_dir = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.566543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] log_file = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.566777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.567022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.567291 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.567535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.567709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.567868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.568115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.568358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.568595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.568824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.569052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.569282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.569526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.569768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.570043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.570549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.570762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] publish_errors = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.570950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.571167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.571354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.571584 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.571803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.572035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] standard_threads = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.572268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.572509 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.572697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.572885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] use_journal = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.573079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] use_json = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.573262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] use_stderr = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.573444 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] use_syslog = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.573639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:33:28.573820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.574006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.574256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.574498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.575156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.575156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.575518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.575702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.576013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.576173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.576432 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.576667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.576917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.577155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.577402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.577659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.577898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.578154 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.578408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.578660 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.578898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.579178 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.579448 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.579702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.579947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.580180 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.580412 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.580644 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.580880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.581131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.581377 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.581606 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.581837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.582083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.582337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.582569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.582850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.583102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.583325 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.583511 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.583748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.583979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.584209 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.584441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.584680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.584934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.585168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.585405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.585634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.585873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.586108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.586356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.586586 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.586831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.587068 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.587361 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.587547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.587781 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.588013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.588265 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.588507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.588746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.588980 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.589243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.589480 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.589713 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.589952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.590180 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.590953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.591236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.591486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.591736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.591973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.592210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.592446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.592695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.592932 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.593168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.593411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.593649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.593897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.594132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.594368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.594602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.594836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.595073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.595318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.595503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.595691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.595877 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.596066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.596066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.596442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.596638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.596822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.597006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.597189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.597388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.597603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.597835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.598070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.598298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.598547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.598782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.599029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.599288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.599546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.599804 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.600060 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.600295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] auth.external = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.600528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.600825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.601056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.601286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] auth.password = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.601516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] auth.token = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.601757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.601995 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.602241 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.602472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.602727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.602973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.603212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.603459 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] credential.caching = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.603703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.603954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.604201 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.604457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.604702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.604938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.605186 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.605437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.605675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.606086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.606842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.607096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.607393 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.607638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.607898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.608136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] federation.caching = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.608384 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.608629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.608879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.609113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.609361 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.609667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.609851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.610065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.610322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.610560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.610808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.611045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.caching = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.611352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.611600 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.611836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.612072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.612319 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.612550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.612789 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.613174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.613430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.613665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.613898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.614131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.614379 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.614629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.614958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.615245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.615496 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.615770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.616012 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.616253 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.616499 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.616739 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.616971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.617210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.617456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.617696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.617948 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.618179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.618431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.618679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.618916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.619196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.619432 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.619663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.619914 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.620144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.620386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.620620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.620869 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.621105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.621344 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.621617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.621856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.622091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.622320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.622570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.622815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.623051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.623313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.623552 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.623782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.624027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.624285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.624515 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.624763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.625009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.625252 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.625486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.625716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.625955 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.626188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.626422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.626723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.626919 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.627114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.627389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.627635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.627877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.628106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.628360 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.628595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.628829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.629064 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.629299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.629545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.629790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.630026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.630277 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.630509 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.630756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.630990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.631253 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.631490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.631740 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.631984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.632214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.632453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.632698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] resource.caching = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.632947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.633195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.633431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.633681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.633929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.634166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.634413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.634648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.634887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.635173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] role.caching = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.635408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] role.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.635639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.635892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.636139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.636389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.636642 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.636889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.637137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.637418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.637645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.637879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.638131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.638377 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.638621 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.638864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.639331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.639567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.639817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.640065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.640315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.640832 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.641069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.641322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.641575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.641817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.642037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.642271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.642516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.642770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.643029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.643302 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.643539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.643764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.643997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.644236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] token.caching = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.644471 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.644719 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.644957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.645214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.645463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.645700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.645945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.646190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.646438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.646676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.646916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.647211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.647422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.647713 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.647943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.648193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.648461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.648723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.connection = **** {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.648967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.649211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.649449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.649691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.649927 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.650165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.650400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.650636 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.650879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.651116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.651384 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.651619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.651866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.652101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.652355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.652586 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.653464 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.653748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:33:28.653931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 09:33:28.661495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.681545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.681825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.681965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.691017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.709859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.710121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.713782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.714077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.749017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.790319 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.790587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.850423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.850730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.851001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.851316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.851605 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.851886 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.872729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.872999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.873195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.873364 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.873605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.873767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.881160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.881388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.881645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.881947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.882234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.882521 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.882807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.886993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.887777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.888016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.888234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.888451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.888662 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.888875 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.896345 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.897033 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.911189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.911445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.911640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.911863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.912077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.912287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.912498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.931066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.931373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 09:33:28.931649 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.931826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 09:33:28.932106 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.932367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.932625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.932879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.943173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.943437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.943704 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.943984 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.952273 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:28.955576 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:28.955862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.956148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.956428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.956708 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.956964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.957233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.957505 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.957774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.958093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.958307 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.983638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.985103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:28.985382 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.990055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:28.990345 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.990630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:28.995519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.995746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.995960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.996182 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.996399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.996745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.997465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:33:28.997848 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:28.998152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.002164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.002457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.002741 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.003107 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.003404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.005598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.005889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.006171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.007643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.007849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 09:33:29.008202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.008399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 09:33:29.008676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.008939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.009201 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.009458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.020214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.020505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.020777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.021063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.022788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.023082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.026956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.028722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.030787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.031085 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.031654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.031939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.032221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.032503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.032786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.033051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.033319 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.033592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.033862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.034139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.034404 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.035452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.038585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.041701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.041999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.042418 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.042569 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.042841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.043254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.057242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.058766 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.059055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.059477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.059791 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.060058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.060322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.063765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.064187 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.064472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.067098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.068809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.070351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.070623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.071019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.071347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.073342 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.073630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.073936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.074217 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.074503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.075993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.076296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.076583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.077004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.077280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.077558 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.077830 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.078101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.086489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.086795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.087077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.087381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.087653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.087923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.088184 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.094543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.094829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.098752 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.101916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.102328 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.104165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.107237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.107819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.110533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.110700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.112170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.113625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.113900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.114048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.114214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.114374 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.114839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.115235 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.115587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.115947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.120671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.122719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.125627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.125924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.126205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.126475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.134505 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.134705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.134856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.135000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.135182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.135378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.135713 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.143167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.143464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.143749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.143954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.144103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.144280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.144554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.144836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.145123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.145397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.145663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.145927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.146184 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.146455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.146719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.147701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.149364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.149709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.152492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.152977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.154827 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.155296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.155650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.167803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.168190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.168963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.169364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.169749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.170121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.170481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.182090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55742edec9c0 pid: 93532 (default app) Aug 31 09:33:29.186494 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cff4e-41ba-4338-96e9-2cb4583b3564 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.186783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cff4e-41ba-4338-96e9-2cb4583b3564 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.187057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cff4e-41ba-4338-96e9-2cb4583b3564 None None] PATH_INFO: `/v3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.188392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.191525 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Wed Aug 31 09:33:29 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:29.191925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.196666 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.198674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.201428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.201720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.202027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.208804 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.210394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.213594 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.213901 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.214189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.214470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.223486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.223873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.224088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.231850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:33:29.232192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.232428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.232704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.232979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.233255 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.233538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.233816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.234094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.234366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.234639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.234913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.235204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.235475 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:33:29.269720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55742edec9c0 pid: 93533 (default app) Aug 31 09:33:36.292771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49868068-0b16-4b4f-9433-6389e7aa2456 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.293328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49868068-0b16-4b4f-9433-6389e7aa2456 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.293328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49868068-0b16-4b4f-9433-6389e7aa2456 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.295963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 2/2] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:36.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee90abd-602c-49c3-a519-20617320e34f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.304988 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee90abd-602c-49c3-a519-20617320e34f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.304988 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee90abd-602c-49c3-a519-20617320e34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.377639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: INFO dbcounter [None req-3ee90abd-602c-49c3-a519-20617320e34f None None] Registered counter for database keystone Aug 31 09:33:36.399316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG oslo_db.sqlalchemy.engines [None req-3ee90abd-602c-49c3-a519-20617320e34f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93533) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 09:33:36.406525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writer thread running {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 09:33:36.594497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG passlib.handlers.bcrypt [None req-3ee90abd-602c-49c3-a519-20617320e34f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93533) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 09:33:36.594871 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG passlib.handlers.bcrypt [None req-3ee90abd-602c-49c3-a519-20617320e34f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93533) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 09:33:36.661364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-3ee90abd-602c-49c3-a519-20617320e34f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:36.662940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3ee90abd-602c-49c3-a519-20617320e34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:36.810773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1/3] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 510 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 09:33:37.209685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49868068-0b16-4b4f-9433-6389e7aa2456 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:37.281404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: INFO dbcounter [None req-49868068-0b16-4b4f-9433-6389e7aa2456 None None] Registered counter for database keystone Aug 31 09:33:37.303546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG oslo_db.sqlalchemy.engines [None req-49868068-0b16-4b4f-9433-6389e7aa2456 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93532) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 09:33:37.310802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writer thread running {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 09:33:37.442073 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49868068-0b16-4b4f-9433-6389e7aa2456 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:37.446466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ab8d7-3c82-41c9-9a0f-7c0e57628a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:37.447113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ab8d7-3c82-41c9-9a0f-7c0e57628a6a admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.447430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ab8d7-3c82-41c9-9a0f-7c0e57628a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.447706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ab8d7-3c82-41c9-9a0f-7c0e57628a6a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.458004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ab8d7-3c82-41c9-9a0f-7c0e57628a6a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:37.510085 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eb4ab8d7-3c82-41c9-9a0f-7c0e57628a6a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 09:33:37.510427 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ab8d7-3c82-41c9-9a0f-7c0e57628a6a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:37.521834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 3/4] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Aug 31 09:33:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:33:37.532771 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee90abd-602c-49c3-a519-20617320e34f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:37.560017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee90abd-602c-49c3-a519-20617320e34f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:37.564814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835c027-3c01-45c1-8ce2-63b925e996b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:37.566194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835c027-3c01-45c1-8ce2-63b925e996b3 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.566525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835c027-3c01-45c1-8ce2-63b925e996b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.566852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835c027-3c01-45c1-8ce2-63b925e996b3 admin admin] PATH_INFO: `/v3/projects/d473eacf42fc47c2aa82e80873c7637e` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.568490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835c027-3c01-45c1-8ce2-63b925e996b3 admin admin] RBAC: Authorizing `identity:get_project(project_id=d473eacf42fc47c2aa82e80873c7637e)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:37.620696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5835c027-3c01-45c1-8ce2-63b925e996b3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 09:33:37.621122 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835c027-3c01-45c1-8ce2-63b925e996b3 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:37.629023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 2/5] 10.222.0.36 () {56 vars in 1207 bytes} [Wed Aug 31 09:33:37 2022] GET /identity/v3/projects/d473eacf42fc47c2aa82e80873c7637e => generated 344 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:39.677630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2f844-b3ab-42fd-8b11-74adc8351a30 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.678229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2f844-b3ab-42fd-8b11-74adc8351a30 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.678229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2f844-b3ab-42fd-8b11-74adc8351a30 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.682342 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 4/6] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:39.689238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24beec9-b1eb-481e-9511-b7dd61c3c6d8 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.689817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24beec9-b1eb-481e-9511-b7dd61c3c6d8 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.689817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24beec9-b1eb-481e-9511-b7dd61c3c6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.710892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f24beec9-b1eb-481e-9511-b7dd61c3c6d8 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:39.712477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f24beec9-b1eb-481e-9511-b7dd61c3c6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:39.721785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 3/7] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 31 09:33:40.001420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2f844-b3ab-42fd-8b11-74adc8351a30 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.014307 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2f844-b3ab-42fd-8b11-74adc8351a30 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.018939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146872e0-91c4-45c5-bd80-9e2c5cb9d23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.019613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146872e0-91c4-45c5-bd80-9e2c5cb9d23d admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.019930 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146872e0-91c4-45c5-bd80-9e2c5cb9d23d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.020239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146872e0-91c4-45c5-bd80-9e2c5cb9d23d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.029538 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146872e0-91c4-45c5-bd80-9e2c5cb9d23d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:40.031404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146872e0-91c4-45c5-bd80-9e2c5cb9d23d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.041413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 5/8] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 09:33:40.048231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24beec9-b1eb-481e-9511-b7dd61c3c6d8 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.058297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24beec9-b1eb-481e-9511-b7dd61c3c6d8 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.063444 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51b657-03b9-402b-832e-42b066b64b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.064106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51b657-03b9-402b-832e-42b066b64b54 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.064380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51b657-03b9-402b-832e-42b066b64b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.064646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51b657-03b9-402b-832e-42b066b64b54 admin admin] PATH_INFO: `/v3/users/d7063e9478d747e983e7ced895545d44` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.066106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51b657-03b9-402b-832e-42b066b64b54 admin admin] RBAC: Authorizing `identity:get_user(user_id=d7063e9478d747e983e7ced895545d44)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:40.068059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51b657-03b9-402b-832e-42b066b64b54 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.069785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 4/9] 10.222.0.36 () {56 vars in 1198 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/users/d7063e9478d747e983e7ced895545d44 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:42.179996 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87b044-e948-4a16-b634-d5ab758c8fe9 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.180778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87b044-e948-4a16-b634-d5ab758c8fe9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.180778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87b044-e948-4a16-b634-d5ab758c8fe9 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.181890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 6/10] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:42.189750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d102b-aa02-4337-b7a3-fa428ce649e2 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.190549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d102b-aa02-4337-b7a3-fa428ce649e2 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.190549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d102b-aa02-4337-b7a3-fa428ce649e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.210256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-220d102b-aa02-4337-b7a3-fa428ce649e2 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:42.211702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-220d102b-aa02-4337-b7a3-fa428ce649e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:42.228029 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e82de-b596-4997-a990-97bcd4c76f86 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.228481 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e82de-b596-4997-a990-97bcd4c76f86 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.228935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e82de-b596-4997-a990-97bcd4c76f86 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.230368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 7/11] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:42.239279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac32f2c-d998-4706-b357-2e03d912b844 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.239703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac32f2c-d998-4706-b357-2e03d912b844 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.240081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac32f2c-d998-4706-b357-2e03d912b844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.286448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG passlib.handlers.bcrypt [None req-9ac32f2c-d998-4706-b357-2e03d912b844 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93532) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 09:33:42.286927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG passlib.handlers.bcrypt [None req-9ac32f2c-d998-4706-b357-2e03d912b844 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93532) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 09:33:42.308383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 5/12] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:42.337093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-9ac32f2c-d998-4706-b357-2e03d912b844 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:42.338660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9ac32f2c-d998-4706-b357-2e03d912b844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:42.452687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 8/13] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:42.586004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d102b-aa02-4337-b7a3-fa428ce649e2 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.629277 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.732986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac32f2c-d998-4706-b357-2e03d912b844 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.759945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d102b-aa02-4337-b7a3-fa428ce649e2 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.800124 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a37e2d-0020-42e3-9a09-728c79e2382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.800948 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a37e2d-0020-42e3-9a09-728c79e2382f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.801312 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a37e2d-0020-42e3-9a09-728c79e2382f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.801683 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a37e2d-0020-42e3-9a09-728c79e2382f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.816349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a37e2d-0020-42e3-9a09-728c79e2382f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-15a37e2d-0020-42e3-9a09-728c79e2382f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:42.831763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:42.832930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:33:42.835724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 6/14] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:42.840793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a37e2d-0020-42e3-9a09-728c79e2382f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.870010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac32f2c-d998-4706-b357-2e03d912b844 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.875149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.921867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100c8b4-8325-4fab-a43c-e1464290b419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.922737 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100c8b4-8325-4fab-a43c-e1464290b419 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.923068 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100c8b4-8325-4fab-a43c-e1464290b419 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.923584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100c8b4-8325-4fab-a43c-e1464290b419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.971064 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.996766 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a37e2d-0020-42e3-9a09-728c79e2382f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.015397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100c8b4-8325-4fab-a43c-e1464290b419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.017592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100c8b4-8325-4fab-a43c-e1464290b419 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.042676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2240f-24f7-42b2-9c30-afdac3728e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.043338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2240f-24f7-42b2-9c30-afdac3728e06 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.044170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2240f-24f7-42b2-9c30-afdac3728e06 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.044490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2240f-24f7-42b2-9c30-afdac3728e06 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.045223 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2240f-24f7-42b2-9c30-afdac3728e06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:43.045746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2240f-24f7-42b2-9c30-afdac3728e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.048497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2240f-24f7-42b2-9c30-afdac3728e06 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.075740 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 7/15] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:43.082295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2240f-24f7-42b2-9c30-afdac3728e06 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.144920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 9/16] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:33:43.154459 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100c8b4-8325-4fab-a43c-e1464290b419 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.158243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.226620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2240f-24f7-42b2-9c30-afdac3728e06 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.267615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99bad-b1e8-4b73-9840-b872d9d63288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.268697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99bad-b1e8-4b73-9840-b872d9d63288 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.269197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99bad-b1e8-4b73-9840-b872d9d63288 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.269674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99bad-b1e8-4b73-9840-b872d9d63288 None admin] PATH_INFO: `/v3/users/d7063e9478d747e983e7ced895545d44` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.271715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec99bad-b1e8-4b73-9840-b872d9d63288 None admin] RBAC: Authorizing `identity:get_user(user_id=d7063e9478d747e983e7ced895545d44)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.274455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec99bad-b1e8-4b73-9840-b872d9d63288 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.277468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 8/17] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/users/d7063e9478d747e983e7ced895545d44 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:43.286601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99bad-b1e8-4b73-9840-b872d9d63288 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.300427 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100c8b4-8325-4fab-a43c-e1464290b419 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.311818 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.347361 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ca582-30de-40c1-b89c-fda8bbecfdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.348771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ca582-30de-40c1-b89c-fda8bbecfdaa None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.349257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ca582-30de-40c1-b89c-fda8bbecfdaa None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.349799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ca582-30de-40c1-b89c-fda8bbecfdaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.381200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ca582-30de-40c1-b89c-fda8bbecfdaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.383720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ca582-30de-40c1-b89c-fda8bbecfdaa None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.401157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99bad-b1e8-4b73-9840-b872d9d63288 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-9a4ca582-30de-40c1-b89c-fda8bbecfdaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:43.417193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:43.418126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:43.419980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:43.419980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:43.419980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:33:43.419980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 10/18] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:43.429174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ca582-30de-40c1-b89c-fda8bbecfdaa None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.436075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b56c13-7f91-48dd-b049-f713a4984783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.436721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b56c13-7f91-48dd-b049-f713a4984783 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.436991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b56c13-7f91-48dd-b049-f713a4984783 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.437264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b56c13-7f91-48dd-b049-f713a4984783 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.438467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b56c13-7f91-48dd-b049-f713a4984783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.440237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b56c13-7f91-48dd-b049-f713a4984783 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.441897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 9/19] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:43.447834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b56c13-7f91-48dd-b049-f713a4984783 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.517289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.540403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.561059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b56c13-7f91-48dd-b049-f713a4984783 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.567411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ca582-30de-40c1-b89c-fda8bbecfdaa None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.602525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47d0da-a904-4ba8-b4c4-b09669dafee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.603534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47d0da-a904-4ba8-b4c4-b09669dafee4 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.604003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47d0da-a904-4ba8-b4c4-b09669dafee4 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.604463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47d0da-a904-4ba8-b4c4-b09669dafee4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.605557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a9581-eb3a-4e5a-8956-3e8f575072fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.605672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47d0da-a904-4ba8-b4c4-b09669dafee4 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.domain.id=default, user.id=d7063e9478d747e983e7ced895545d44) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:43.606773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47d0da-a904-4ba8-b4c4-b09669dafee4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.607058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a9581-eb3a-4e5a-8956-3e8f575072fc None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.607551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a9581-eb3a-4e5a-8956-3e8f575072fc None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.607968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a9581-eb3a-4e5a-8956-3e8f575072fc None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.609066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a9581-eb3a-4e5a-8956-3e8f575072fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:43.609407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47d0da-a904-4ba8-b4c4-b09669dafee4 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.609573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a9581-eb3a-4e5a-8956-3e8f575072fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.611949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a9581-eb3a-4e5a-8956-3e8f575072fc None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.635155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 10/20] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/role_assignments?user.id=d7063e9478d747e983e7ced895545d44&scope.domain.id=default&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 233 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:43.635521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 11/21] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:45.738931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f7d9a-405f-424a-b4b1-21d821c656b4 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:45.739491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f7d9a-405f-424a-b4b1-21d821c656b4 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:45.739922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f7d9a-405f-424a-b4b1-21d821c656b4 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:45.742566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 11/22] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:45.749577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa6bbd-7eeb-4c3f-80e3-ce876b645d2e None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:45.750082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa6bbd-7eeb-4c3f-80e3-ce876b645d2e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:45.750497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa6bbd-7eeb-4c3f-80e3-ce876b645d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:45.790770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-caaa6bbd-7eeb-4c3f-80e3-ce876b645d2e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:45.792855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-caaa6bbd-7eeb-4c3f-80e3-ce876b645d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:45.946920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 12/23] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:46.172590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224176c5-f37e-4156-bcdc-3e73326e0758 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.173217 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224176c5-f37e-4156-bcdc-3e73326e0758 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.173711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224176c5-f37e-4156-bcdc-3e73326e0758 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.175281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 12/24] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:46.191833 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbb91d-cff9-4be3-a655-0fb0931dd861 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.192462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbb91d-cff9-4be3-a655-0fb0931dd861 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.192938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbb91d-cff9-4be3-a655-0fb0931dd861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.220352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ebdbb91d-cff9-4be3-a655-0fb0931dd861 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:46.221928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ebdbb91d-cff9-4be3-a655-0fb0931dd861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:46.272369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:46.309886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 13/25] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:46.549193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224176c5-f37e-4156-bcdc-3e73326e0758 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:46.690757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff491ab4-05d1-4412-8899-075b9e3b7a0f None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.691328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff491ab4-05d1-4412-8899-075b9e3b7a0f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.691735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff491ab4-05d1-4412-8899-075b9e3b7a0f None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.694467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 14/26] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:46.700339 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1c153-0d90-402a-848a-3ec714bb8fa6 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.700821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1c153-0d90-402a-848a-3ec714bb8fa6 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.701231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1c153-0d90-402a-848a-3ec714bb8fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.722715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224176c5-f37e-4156-bcdc-3e73326e0758 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:46.742322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-d8d1c153-0d90-402a-848a-3ec714bb8fa6 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:46.743794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d8d1c153-0d90-402a-848a-3ec714bb8fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:46.808995 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791a368-cfa4-4375-aa0b-a4800c4bdb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:46.810387 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791a368-cfa4-4375-aa0b-a4800c4bdb48 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.810802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791a368-cfa4-4375-aa0b-a4800c4bdb48 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.811367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791a368-cfa4-4375-aa0b-a4800c4bdb48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.842911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791a368-cfa4-4375-aa0b-a4800c4bdb48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-b791a368-cfa4-4375-aa0b-a4800c4bdb48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:46.848300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:46.849259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:46.850102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:46.850102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:46.850102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:46.850102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:46.850102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:46.850102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:46.850102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:46.850102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:46.850102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:46.850102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:46.850102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:46.850102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:33:46.859161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 13/27] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:46.860442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791a368-cfa4-4375-aa0b-a4800c4bdb48 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:46.880513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:46.886085 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 15/28] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:46.891160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1c153-0d90-402a-848a-3ec714bb8fa6 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.094640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1c153-0d90-402a-848a-3ec714bb8fa6 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.111156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.163242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91cb24-8fa6-4e32-936f-401939ddca11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.164982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91cb24-8fa6-4e32-936f-401939ddca11 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.165449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91cb24-8fa6-4e32-936f-401939ddca11 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.165939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91cb24-8fa6-4e32-936f-401939ddca11 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.167168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91cb24-8fa6-4e32-936f-401939ddca11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:47.167721 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91cb24-8fa6-4e32-936f-401939ddca11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.168227 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791a368-cfa4-4375-aa0b-a4800c4bdb48 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.174652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91cb24-8fa6-4e32-936f-401939ddca11 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.196337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 16/29] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:47.203912 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91cb24-8fa6-4e32-936f-401939ddca11 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.262129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fed27a-11d9-44ff-8935-c4f4b7135c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.264332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fed27a-11d9-44ff-8935-c4f4b7135c1a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.265549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fed27a-11d9-44ff-8935-c4f4b7135c1a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.266963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fed27a-11d9-44ff-8935-c4f4b7135c1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.290272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fed27a-11d9-44ff-8935-c4f4b7135c1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.293964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fed27a-11d9-44ff-8935-c4f4b7135c1a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-36fed27a-11d9-44ff-8935-c4f4b7135c1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:47.328323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:47.329984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:47.332231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:47.332231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:47.332231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:33:47.334984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 14/30] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 475 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:47.337033 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fed27a-11d9-44ff-8935-c4f4b7135c1a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.403466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91cb24-8fa6-4e32-936f-401939ddca11 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.455693 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.469564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c4bb9-53de-4666-9fe0-c06fb335e5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.470225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c4bb9-53de-4666-9fe0-c06fb335e5b1 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.470503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c4bb9-53de-4666-9fe0-c06fb335e5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.470778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c4bb9-53de-4666-9fe0-c06fb335e5b1 None admin] PATH_INFO: `/v3/users/d7063e9478d747e983e7ced895545d44` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.476302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c4bb9-53de-4666-9fe0-c06fb335e5b1 None admin] RBAC: Authorizing `identity:get_user(user_id=d7063e9478d747e983e7ced895545d44)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.477803 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c4bb9-53de-4666-9fe0-c06fb335e5b1 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.479826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 17/31] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/users/d7063e9478d747e983e7ced895545d44 => generated 247 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:47.480256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c4bb9-53de-4666-9fe0-c06fb335e5b1 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.525960 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.560605 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fed27a-11d9-44ff-8935-c4f4b7135c1a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.629339 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d6681-1383-4806-ad20-6972ef5dc011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.630769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d6681-1383-4806-ad20-6972ef5dc011 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d6681-1383-4806-ad20-6972ef5dc011 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.632486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d6681-1383-4806-ad20-6972ef5dc011 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.634130 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d6681-1383-4806-ad20-6972ef5dc011 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.637877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d6681-1383-4806-ad20-6972ef5dc011 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.677599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c4bb9-53de-4666-9fe0-c06fb335e5b1 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.744583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917464ba-00e6-4b3b-b63c-3685b517e97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.745206 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917464ba-00e6-4b3b-b63c-3685b517e97a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.745482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917464ba-00e6-4b3b-b63c-3685b517e97a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.745757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917464ba-00e6-4b3b-b63c-3685b517e97a None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.746486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917464ba-00e6-4b3b-b63c-3685b517e97a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:47.746794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917464ba-00e6-4b3b-b63c-3685b517e97a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.748427 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917464ba-00e6-4b3b-b63c-3685b517e97a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.762587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 18/32] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:47.763142 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917464ba-00e6-4b3b-b63c-3685b517e97a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.873147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.880331 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917464ba-00e6-4b3b-b63c-3685b517e97a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.890095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 15/33] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Aug 31 09:33:47 2022] POST /identity/v3/roles => generated 219 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:47.894449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144caabe-1501-4b2a-9520-e44637dd5751 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.895569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144caabe-1501-4b2a-9520-e44637dd5751 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.896300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144caabe-1501-4b2a-9520-e44637dd5751 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.899782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 16/34] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:47.901534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144caabe-1501-4b2a-9520-e44637dd5751 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.961352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77142b-2d45-4dc8-817a-0aa66e710f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.962132 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77142b-2d45-4dc8-817a-0aa66e710f83 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.962256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77142b-2d45-4dc8-817a-0aa66e710f83 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.962604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77142b-2d45-4dc8-817a-0aa66e710f83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.963825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77142b-2d45-4dc8-817a-0aa66e710f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.968519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77142b-2d45-4dc8-817a-0aa66e710f83 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.970640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 19/35] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:47.971680 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1900196-96ac-4c69-9328-f44eaa8b91f4 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.971935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1900196-96ac-4c69-9328-f44eaa8b91f4 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.978934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1900196-96ac-4c69-9328-f44eaa8b91f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.015959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b1900196-96ac-4c69-9328-f44eaa8b91f4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:48.018168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b1900196-96ac-4c69-9328-f44eaa8b91f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:48.062438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144caabe-1501-4b2a-9520-e44637dd5751 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.090825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.115401 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405eb65-bdee-4419-8abe-497ff8bbd071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.115986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405eb65-bdee-4419-8abe-497ff8bbd071 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.116268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405eb65-bdee-4419-8abe-497ff8bbd071 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.116541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405eb65-bdee-4419-8abe-497ff8bbd071 None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.117231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405eb65-bdee-4419-8abe-497ff8bbd071 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.118959 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405eb65-bdee-4419-8abe-497ff8bbd071 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.125724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 20/36] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:48.126774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1900196-96ac-4c69-9328-f44eaa8b91f4 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.164550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 17/37] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:33:47 2022] POST /identity/v3/projects => generated 301 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:48.169243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb57936-286f-44df-a05a-a7f7cd7f74ee None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.169243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb57936-286f-44df-a05a-a7f7cd7f74ee None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.169243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb57936-286f-44df-a05a-a7f7cd7f74ee None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.169891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 18/38] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:48.176696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466dae5-ed77-442a-a340-1c86eda3c7bb None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.177229 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466dae5-ed77-442a-a340-1c86eda3c7bb None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.177697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466dae5-ed77-442a-a340-1c86eda3c7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.202518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-2466dae5-ed77-442a-a340-1c86eda3c7bb None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:48.204360 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2466dae5-ed77-442a-a340-1c86eda3c7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:48.240864 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.250054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1900196-96ac-4c69-9328-f44eaa8b91f4 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.296113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33544981-91c3-406d-90af-746bbc93c9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.297217 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33544981-91c3-406d-90af-746bbc93c9ad None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.297752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33544981-91c3-406d-90af-746bbc93c9ad None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.298210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33544981-91c3-406d-90af-746bbc93c9ad None admin] PATH_INFO: `/v3/domains/default/users/d7063e9478d747e983e7ced895545d44/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.300712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33544981-91c3-406d-90af-746bbc93c9ad None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d7063e9478d747e983e7ced895545d44, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.302560 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33544981-91c3-406d-90af-746bbc93c9ad None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.344023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 19/39] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:48.351695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cb5f3-4e3a-4628-865c-746ead9bbb4e None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.352313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cb5f3-4e3a-4628-865c-746ead9bbb4e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.352897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cb5f3-4e3a-4628-865c-746ead9bbb4e None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.355882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 20/40] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:48 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:48.365218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de293692-160b-4ade-8f50-e468754f9446 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.365649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de293692-160b-4ade-8f50-e468754f9446 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.366102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de293692-160b-4ade-8f50-e468754f9446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.419749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-de293692-160b-4ade-8f50-e468754f9446 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:48.421844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-de293692-160b-4ade-8f50-e468754f9446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:48.441645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 21/41] 10.222.0.36 () {58 vars in 1361 bytes} [Wed Aug 31 09:33:48 2022] PUT /identity/v3/domains/default/users/d7063e9478d747e983e7ced895545d44/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:48.443277 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33544981-91c3-406d-90af-746bbc93c9ad None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.563965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 21/42] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:48.634340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33544981-91c3-406d-90af-746bbc93c9ad None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.678210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f594ffb-d2c3-439d-9392-36870af660b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.678847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f594ffb-d2c3-439d-9392-36870af660b6 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.679146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f594ffb-d2c3-439d-9392-36870af660b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.679431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f594ffb-d2c3-439d-9392-36870af660b6 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.680127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f594ffb-d2c3-439d-9392-36870af660b6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.681893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f594ffb-d2c3-439d-9392-36870af660b6 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.707661 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 22/43] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Aug 31 09:33:48 2022] POST /identity/v3/roles => generated 225 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:48.825453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de293692-160b-4ade-8f50-e468754f9446 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.895402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.015281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de293692-160b-4ade-8f50-e468754f9446 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.058388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8715807-cf8a-451a-aee4-56ffa722a246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.058388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8715807-cf8a-451a-aee4-56ffa722a246 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.058388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8715807-cf8a-451a-aee4-56ffa722a246 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.058388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8715807-cf8a-451a-aee4-56ffa722a246 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.059452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8715807-cf8a-451a-aee4-56ffa722a246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.060942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8715807-cf8a-451a-aee4-56ffa722a246 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.062961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 22/44] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:49.069125 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f594ffb-d2c3-439d-9392-36870af660b6 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.101186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.132321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8715807-cf8a-451a-aee4-56ffa722a246 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.267082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f594ffb-d2c3-439d-9392-36870af660b6 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.303473 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.318327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8715807-cf8a-451a-aee4-56ffa722a246 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.326221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e003c2-ba75-4ba0-a65e-00f3c6a2c51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.327041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e003c2-ba75-4ba0-a65e-00f3c6a2c51f None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.327480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e003c2-ba75-4ba0-a65e-00f3c6a2c51f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.327894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e003c2-ba75-4ba0-a65e-00f3c6a2c51f None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.328689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e003c2-ba75-4ba0-a65e-00f3c6a2c51f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.330315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e003c2-ba75-4ba0-a65e-00f3c6a2c51f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.368087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 23/45] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:33:49 2022] POST /identity/v3/projects => generated 312 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:49.404696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310ddc6-0495-4def-9508-7df3c5414cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.405817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310ddc6-0495-4def-9508-7df3c5414cc9 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.406307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310ddc6-0495-4def-9508-7df3c5414cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.406773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310ddc6-0495-4def-9508-7df3c5414cc9 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.408206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310ddc6-0495-4def-9508-7df3c5414cc9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.410894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310ddc6-0495-4def-9508-7df3c5414cc9 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.441524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 23/46] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Aug 31 09:33:49 2022] POST /identity/v3/roles => generated 223 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:50.581470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7e9a-7fa6-42bb-9994-78fa47806a53 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.582010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7e9a-7fa6-42bb-9994-78fa47806a53 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.582373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7e9a-7fa6-42bb-9994-78fa47806a53 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.584902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 24/47] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:50.594242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed915d-d689-4ce0-b9dc-afd1df6b190c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.594964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed915d-d689-4ce0-b9dc-afd1df6b190c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.594964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed915d-d689-4ce0-b9dc-afd1df6b190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.615765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-42ed915d-d689-4ce0-b9dc-afd1df6b190c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:50.617024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-42ed915d-d689-4ce0-b9dc-afd1df6b190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:50.701302 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 24/48] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:50.972002 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a7e9a-7fa6-42bb-9994-78fa47806a53 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.052879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.115988 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a7e9a-7fa6-42bb-9994-78fa47806a53 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.181178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c02a5f-383d-43f7-b834-6ab5465740e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.182288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c02a5f-383d-43f7-b834-6ab5465740e4 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.182790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c02a5f-383d-43f7-b834-6ab5465740e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.183337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c02a5f-383d-43f7-b834-6ab5465740e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.207026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c02a5f-383d-43f7-b834-6ab5465740e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-10c02a5f-383d-43f7-b834-6ab5465740e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:51.221245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:51.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:51.223189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:51.223189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:51.223189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:51.223189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:51.223189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:51.223189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:51.223189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:51.223189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:51.223189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:51.223189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:51.223189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:51.223189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:33:51.223981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 25/49] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:51.234062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed915d-d689-4ce0-b9dc-afd1df6b190c None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.461583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.482885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed915d-d689-4ce0-b9dc-afd1df6b190c None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.523708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d23bf7-3b18-4360-a905-aa51983b4edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.524425 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d23bf7-3b18-4360-a905-aa51983b4edc None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.524771 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d23bf7-3b18-4360-a905-aa51983b4edc None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.525114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d23bf7-3b18-4360-a905-aa51983b4edc None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.525877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d23bf7-3b18-4360-a905-aa51983b4edc None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:51.526245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d23bf7-3b18-4360-a905-aa51983b4edc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.527881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d23bf7-3b18-4360-a905-aa51983b4edc None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.537860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 25/50] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:51.544338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c02a5f-383d-43f7-b834-6ab5465740e4 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.664746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c02a5f-383d-43f7-b834-6ab5465740e4 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.685469 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f750755-396c-4241-a739-a58571fc319a None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.685991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f750755-396c-4241-a739-a58571fc319a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.686437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f750755-396c-4241-a739-a58571fc319a None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.689027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 26/51] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:51.694687 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6022f-f49b-4e67-8882-390b1e40aa44 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.694963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6022f-f49b-4e67-8882-390b1e40aa44 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.695453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6022f-f49b-4e67-8882-390b1e40aa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.712565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.715993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-35d6022f-f49b-4e67-8882-390b1e40aa44 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:51.717509 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-35d6022f-f49b-4e67-8882-390b1e40aa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:51.729038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e95887-998f-4cbc-895e-cec50a4c22e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.730158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e95887-998f-4cbc-895e-cec50a4c22e3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.730627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e95887-998f-4cbc-895e-cec50a4c22e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.732136 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e95887-998f-4cbc-895e-cec50a4c22e3 None admin] PATH_INFO: `/v3/users/d7063e9478d747e983e7ced895545d44` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.734417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e95887-998f-4cbc-895e-cec50a4c22e3 None admin] RBAC: Authorizing `identity:get_user(user_id=d7063e9478d747e983e7ced895545d44)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.737089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e95887-998f-4cbc-895e-cec50a4c22e3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.744587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 26/52] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/users/d7063e9478d747e983e7ced895545d44 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:51.747320 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e95887-998f-4cbc-895e-cec50a4c22e3 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.794927 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 27/53] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:52.050709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e95887-998f-4cbc-895e-cec50a4c22e3 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.067344 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6022f-f49b-4e67-8882-390b1e40aa44 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.098004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40f0b9-23a6-47e8-b3ee-e58d6f63cd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.098882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40f0b9-23a6-47e8-b3ee-e58d6f63cd0e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.099271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40f0b9-23a6-47e8-b3ee-e58d6f63cd0e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.099625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40f0b9-23a6-47e8-b3ee-e58d6f63cd0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.101257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40f0b9-23a6-47e8-b3ee-e58d6f63cd0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.102987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40f0b9-23a6-47e8-b3ee-e58d6f63cd0e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.105095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 27/54] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:52.114062 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40f0b9-23a6-47e8-b3ee-e58d6f63cd0e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.168888 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.196207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.196823 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6022f-f49b-4e67-8882-390b1e40aa44 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.237859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21162eb0-fa23-465a-9aa4-e26dcd6064f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.238525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21162eb0-fa23-465a-9aa4-e26dcd6064f5 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.238801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21162eb0-fa23-465a-9aa4-e26dcd6064f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.239067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21162eb0-fa23-465a-9aa4-e26dcd6064f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.240355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21162eb0-fa23-465a-9aa4-e26dcd6064f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.241868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21162eb0-fa23-465a-9aa4-e26dcd6064f5 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.243629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 28/55] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:52.250260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21162eb0-fa23-465a-9aa4-e26dcd6064f5 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.266458 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40f0b9-23a6-47e8-b3ee-e58d6f63cd0e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.319256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70961be7-5ca1-4158-b687-b140c4e3c038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.319774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70961be7-5ca1-4158-b687-b140c4e3c038 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.320075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70961be7-5ca1-4158-b687-b140c4e3c038 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.320348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70961be7-5ca1-4158-b687-b140c4e3c038 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.321099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70961be7-5ca1-4158-b687-b140c4e3c038 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.domain.id=default, user.id=d7063e9478d747e983e7ced895545d44) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:52.321410 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70961be7-5ca1-4158-b687-b140c4e3c038 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.322898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70961be7-5ca1-4158-b687-b140c4e3c038 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.332298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 28/56] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/role_assignments?user.id=d7063e9478d747e983e7ced895545d44&scope.domain.id=default&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 530 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:52.375519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21162eb0-fa23-465a-9aa4-e26dcd6064f5 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.420121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77f1fd-95a7-4556-8aea-aa957b6ba674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.420908 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f1fd-95a7-4556-8aea-aa957b6ba674 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.421328 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f1fd-95a7-4556-8aea-aa957b6ba674 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.421688 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f1fd-95a7-4556-8aea-aa957b6ba674 None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.422476 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77f1fd-95a7-4556-8aea-aa957b6ba674 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.424143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77f1fd-95a7-4556-8aea-aa957b6ba674 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.449380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.458343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 29/57] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Aug 31 09:33:52 2022] POST /identity/v3/projects => generated 298 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:33:54.486686 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc0a68-746f-4a3b-b3d2-79f3bbce32d1 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.487299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc0a68-746f-4a3b-b3d2-79f3bbce32d1 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.487299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc0a68-746f-4a3b-b3d2-79f3bbce32d1 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.488652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 29/58] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:54.494377 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aeb63a-e758-459f-b2b4-b1938e24a837 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.494865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aeb63a-e758-459f-b2b4-b1938e24a837 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.495017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aeb63a-e758-459f-b2b4-b1938e24a837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.514715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b1aeb63a-e758-459f-b2b4-b1938e24a837 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:54.516225 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b1aeb63a-e758-459f-b2b4-b1938e24a837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:54.588685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 30/59] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:54.885017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc0a68-746f-4a3b-b3d2-79f3bbce32d1 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.998265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc0a68-746f-4a3b-b3d2-79f3bbce32d1 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:55.026575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:55.037595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db05f5-8e53-49c9-998a-a6b659122ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:55.038252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db05f5-8e53-49c9-998a-a6b659122ce1 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.038533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db05f5-8e53-49c9-998a-a6b659122ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.038808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db05f5-8e53-49c9-998a-a6b659122ce1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.040184 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db05f5-8e53-49c9-998a-a6b659122ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:55.041632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db05f5-8e53-49c9-998a-a6b659122ce1 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:55.043304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 30/60] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:55.049697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aeb63a-e758-459f-b2b4-b1938e24a837 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:55.159514 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aeb63a-e758-459f-b2b4-b1938e24a837 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:55.188666 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:55.198604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893b231-02c4-4f30-b909-e4012fcfe087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:55.199325 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893b231-02c4-4f30-b909-e4012fcfe087 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.200143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893b231-02c4-4f30-b909-e4012fcfe087 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.200399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893b231-02c4-4f30-b909-e4012fcfe087 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.201068 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893b231-02c4-4f30-b909-e4012fcfe087 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:55.202542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893b231-02c4-4f30-b909-e4012fcfe087 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:55.232602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 31/61] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:55 2022] POST /identity/v3/users => generated 275 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:59.083168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695aade-d3b5-430d-9196-d493ef4b8539 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.083168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695aade-d3b5-430d-9196-d493ef4b8539 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.083168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695aade-d3b5-430d-9196-d493ef4b8539 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.083168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 31/62] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:59.093283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9015ed3-68fe-427a-b4b5-400239f7a520 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.094627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9015ed3-68fe-427a-b4b5-400239f7a520 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.095410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9015ed3-68fe-427a-b4b5-400239f7a520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.148773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a9015ed3-68fe-427a-b4b5-400239f7a520 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:59.151216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a9015ed3-68fe-427a-b4b5-400239f7a520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:59.294583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 32/63] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:59.553548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72340811-4048-4df9-995c-81f8b8e88fd5 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.554043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72340811-4048-4df9-995c-81f8b8e88fd5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.554486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72340811-4048-4df9-995c-81f8b8e88fd5 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.561554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 32/64] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:33:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:59.567932 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb56a0d-4898-4fd8-a195-957e26bf82db None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.568414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb56a0d-4898-4fd8-a195-957e26bf82db None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.568899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb56a0d-4898-4fd8-a195-957e26bf82db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.608988 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-0cb56a0d-4898-4fd8-a195-957e26bf82db None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:59.611024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0cb56a0d-4898-4fd8-a195-957e26bf82db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:59.740662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 33/65] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:59.962035 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72340811-4048-4df9-995c-81f8b8e88fd5 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.095333 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72340811-4048-4df9-995c-81f8b8e88fd5 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.103048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb56a0d-4898-4fd8-a195-957e26bf82db None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.160184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.211320 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c67d5-cd3d-4318-86b1-690d47075a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.212381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c67d5-cd3d-4318-86b1-690d47075a51 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.212773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c67d5-cd3d-4318-86b1-690d47075a51 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.213237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c67d5-cd3d-4318-86b1-690d47075a51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.230205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb56a0d-4898-4fd8-a195-957e26bf82db None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.254496 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c67d5-cd3d-4318-86b1-690d47075a51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-3d3c67d5-cd3d-4318-86b1-690d47075a51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:00.266050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:00.267789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:00.269304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:00.269304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:00.269304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:00.269304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:00.269304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:00.269304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:00.269304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:00.269304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:00.269304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:00.269304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:00.269304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:00.269304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:00.273524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 33/66] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:00.279156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca46969-ac38-44cf-9f76-77544510de40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.279156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca46969-ac38-44cf-9f76-77544510de40 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.279156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca46969-ac38-44cf-9f76-77544510de40 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.279156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca46969-ac38-44cf-9f76-77544510de40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.285614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c67d5-cd3d-4318-86b1-690d47075a51 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.289263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca46969-ac38-44cf-9f76-77544510de40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-dca46969-ac38-44cf-9f76-77544510de40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:00.296388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:00.298034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 34/67] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:00.299640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7243eb2-c64c-4d95-bb72-10c0ef62313e None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.300964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7243eb2-c64c-4d95-bb72-10c0ef62313e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.300964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7243eb2-c64c-4d95-bb72-10c0ef62313e None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.302010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 35/68] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:00.308023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9518d19-2bd4-4569-b41a-2a79c756bf8d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.308443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9518d19-2bd4-4569-b41a-2a79c756bf8d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.309427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9518d19-2bd4-4569-b41a-2a79c756bf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.339348 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f9518d19-2bd4-4569-b41a-2a79c756bf8d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:00.342073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f9518d19-2bd4-4569-b41a-2a79c756bf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:00.368877 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.447523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c67d5-cd3d-4318-86b1-690d47075a51 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.447780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 36/69] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:00.450057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9518d19-2bd4-4569-b41a-2a79c756bf8d None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.512019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c012b-8bd3-447c-a0c7-c4a01099aa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.512983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c012b-8bd3-447c-a0c7-c4a01099aa4e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.513470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c012b-8bd3-447c-a0c7-c4a01099aa4e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.513948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c012b-8bd3-447c-a0c7-c4a01099aa4e None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.515175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c012b-8bd3-447c-a0c7-c4a01099aa4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:00.515934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c012b-8bd3-447c-a0c7-c4a01099aa4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.516764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.518514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c012b-8bd3-447c-a0c7-c4a01099aa4e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.530580 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 34/70] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:00.533336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd268ce-68cd-4f32-9e3b-ffa88549b682 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.533843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd268ce-68cd-4f32-9e3b-ffa88549b682 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.534328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd268ce-68cd-4f32-9e3b-ffa88549b682 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.536921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 35/71] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:00.538941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33809e11-57df-423c-8722-c2d7808d6194 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.539879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33809e11-57df-423c-8722-c2d7808d6194 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.539987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33809e11-57df-423c-8722-c2d7808d6194 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.542513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 36/72] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:00.548644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33809e11-57df-423c-8722-c2d7808d6194 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.651793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9518d19-2bd4-4569-b41a-2a79c756bf8d None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.696210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33809e11-57df-423c-8722-c2d7808d6194 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.709935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2e200-ba0c-4af3-92e1-d64e12d80a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.710714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2e200-ba0c-4af3-92e1-d64e12d80a89 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.711262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2e200-ba0c-4af3-92e1-d64e12d80a89 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.711690 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2e200-ba0c-4af3-92e1-d64e12d80a89 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.712831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2e200-ba0c-4af3-92e1-d64e12d80a89 None admin] RBAC: Adding query filter params (name=member) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:00.713326 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2e200-ba0c-4af3-92e1-d64e12d80a89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.715487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2e200-ba0c-4af3-92e1-d64e12d80a89 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.728048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.729562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 37/73] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:00.732292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55132dd-1577-4606-b245-cb5778ebdb9f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.732777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55132dd-1577-4606-b245-cb5778ebdb9f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.733248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55132dd-1577-4606-b245-cb5778ebdb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.764086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b55132dd-1577-4606-b245-cb5778ebdb9f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:00.766252 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b55132dd-1577-4606-b245-cb5778ebdb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:00.769667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c58441-9f56-4ac3-9d67-bd4f85446ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.770538 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c58441-9f56-4ac3-9d67-bd4f85446ea2 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.771001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c58441-9f56-4ac3-9d67-bd4f85446ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.771466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c58441-9f56-4ac3-9d67-bd4f85446ea2 None admin] PATH_INFO: `/v3/users/d7063e9478d747e983e7ced895545d44` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.773385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c58441-9f56-4ac3-9d67-bd4f85446ea2 None admin] RBAC: Authorizing `identity:get_user(user_id=d7063e9478d747e983e7ced895545d44)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.775920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c58441-9f56-4ac3-9d67-bd4f85446ea2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.778761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 37/74] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/users/d7063e9478d747e983e7ced895545d44 => generated 247 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:00.780756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff1453-2eab-43f9-9d53-56b2ee27bea0 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.781213 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff1453-2eab-43f9-9d53-56b2ee27bea0 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.781664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff1453-2eab-43f9-9d53-56b2ee27bea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.804448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b1ff1453-2eab-43f9-9d53-56b2ee27bea0 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:00.805520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b1ff1453-2eab-43f9-9d53-56b2ee27bea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:00.865972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 38/75] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:00.867474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55132dd-1577-4606-b245-cb5778ebdb9f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.900242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.914088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 38/76] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:00.914957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff1453-2eab-43f9-9d53-56b2ee27bea0 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.022587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55132dd-1577-4606-b245-cb5778ebdb9f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.095717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ae9a4-ec8b-427b-a80d-180dc7367fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.096317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ae9a4-ec8b-427b-a80d-180dc7367fd0 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.096763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ae9a4-ec8b-427b-a80d-180dc7367fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.097177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ae9a4-ec8b-427b-a80d-180dc7367fd0 None admin] PATH_INFO: `/v3/users/dd9b2ddbfdf942fbab0323fe249d8639` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.103959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff1453-2eab-43f9-9d53-56b2ee27bea0 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.116282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ae9a4-ec8b-427b-a80d-180dc7367fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=dd9b2ddbfdf942fbab0323fe249d8639)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.118491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ae9a4-ec8b-427b-a80d-180dc7367fd0 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.120760 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 39/77] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/users/dd9b2ddbfdf942fbab0323fe249d8639 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:01.122603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ae9a4-ec8b-427b-a80d-180dc7367fd0 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.148133 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.178773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e458a-be3f-4c73-9bf7-fa80a2289736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.180176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e458a-be3f-4c73-9bf7-fa80a2289736 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.180573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e458a-be3f-4c73-9bf7-fa80a2289736 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.180961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e458a-be3f-4c73-9bf7-fa80a2289736 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.182782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e458a-be3f-4c73-9bf7-fa80a2289736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.185695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e458a-be3f-4c73-9bf7-fa80a2289736 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.195903 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 39/78] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:01.197858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e458a-be3f-4c73-9bf7-fa80a2289736 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.242612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.265082 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ae9a4-ec8b-427b-a80d-180dc7367fd0 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.310520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f621ee-6c51-4a95-881d-cd8373a6483d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.311534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f621ee-6c51-4a95-881d-cd8373a6483d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.312022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f621ee-6c51-4a95-881d-cd8373a6483d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.312524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f621ee-6c51-4a95-881d-cd8373a6483d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.314963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e458a-be3f-4c73-9bf7-fa80a2289736 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.353033 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950da0f4-b9f2-4676-8efd-7b6167bdd30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.353799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950da0f4-b9f2-4676-8efd-7b6167bdd30e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.354180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950da0f4-b9f2-4676-8efd-7b6167bdd30e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.354618 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950da0f4-b9f2-4676-8efd-7b6167bdd30e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.412201 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f621ee-6c51-4a95-881d-cd8373a6483d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.414062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f621ee-6c51-4a95-881d-cd8373a6483d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.421181 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.444574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950da0f4-b9f2-4676-8efd-7b6167bdd30e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.446908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950da0f4-b9f2-4676-8efd-7b6167bdd30e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.547377 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.573072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 40/79] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:01.573072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f621ee-6c51-4a95-881d-cd8373a6483d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.588571 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 40/80] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:01.589815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950da0f4-b9f2-4676-8efd-7b6167bdd30e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.690365 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f621ee-6c51-4a95-881d-cd8373a6483d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.710890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.729662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc232b0-4b0d-4a96-bba2-77130e2bec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.730314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc232b0-4b0d-4a96-bba2-77130e2bec43 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.730666 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc232b0-4b0d-4a96-bba2-77130e2bec43 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.730927 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc232b0-4b0d-4a96-bba2-77130e2bec43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.733814 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950da0f4-b9f2-4676-8efd-7b6167bdd30e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.742424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc232b0-4b0d-4a96-bba2-77130e2bec43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-4dc232b0-4b0d-4a96-bba2-77130e2bec43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:01.748279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:01.750171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:01.751757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 41/81] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:01.752978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc232b0-4b0d-4a96-bba2-77130e2bec43 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.778274 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6082702-6532-471b-aa5d-6a3eff95f346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.778922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6082702-6532-471b-aa5d-6a3eff95f346 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.779230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6082702-6532-471b-aa5d-6a3eff95f346 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.779509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6082702-6532-471b-aa5d-6a3eff95f346 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.780188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6082702-6532-471b-aa5d-6a3eff95f346 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.781685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6082702-6532-471b-aa5d-6a3eff95f346 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.811811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 41/82] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:01 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:01.812686 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6082702-6532-471b-aa5d-6a3eff95f346 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.825362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.889059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc232b0-4b0d-4a96-bba2-77130e2bec43 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.931751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c39629-8dc8-4520-bbc4-9b863bbf2a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.932414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c39629-8dc8-4520-bbc4-9b863bbf2a9a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.932699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c39629-8dc8-4520-bbc4-9b863bbf2a9a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.932974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c39629-8dc8-4520-bbc4-9b863bbf2a9a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.937171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6082702-6532-471b-aa5d-6a3eff95f346 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.946751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c39629-8dc8-4520-bbc4-9b863bbf2a9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-a1c39629-8dc8-4520-bbc4-9b863bbf2a9a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:01.952342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:01.954070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 42/83] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:01.955619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c39629-8dc8-4520-bbc4-9b863bbf2a9a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.005676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd70e3-f036-4190-bf39-3e611f6ed939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.006342 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd70e3-f036-4190-bf39-3e611f6ed939 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.006635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd70e3-f036-4190-bf39-3e611f6ed939 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.006920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd70e3-f036-4190-bf39-3e611f6ed939 None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.008446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd70e3-f036-4190-bf39-3e611f6ed939 None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.009922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd70e3-f036-4190-bf39-3e611f6ed939 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.018429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 42/84] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.018429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd70e3-f036-4190-bf39-3e611f6ed939 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.023555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.112534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c39629-8dc8-4520-bbc4-9b863bbf2a9a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.117458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.167823 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a1cba-5e3b-4e64-92e6-5fed9d9bef32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.169527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a1cba-5e3b-4e64-92e6-5fed9d9bef32 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.169963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a1cba-5e3b-4e64-92e6-5fed9d9bef32 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.170326 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a1cba-5e3b-4e64-92e6-5fed9d9bef32 None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.171830 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a1cba-5e3b-4e64-92e6-5fed9d9bef32 None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.173733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a1cba-5e3b-4e64-92e6-5fed9d9bef32 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.175077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd70e3-f036-4190-bf39-3e611f6ed939 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.176066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 43/85] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.177053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a1cba-5e3b-4e64-92e6-5fed9d9bef32 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.234183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ef5db-3f90-489f-aacb-ab327bf3535c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.235074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ef5db-3f90-489f-aacb-ab327bf3535c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.235398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ef5db-3f90-489f-aacb-ab327bf3535c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.235732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ef5db-3f90-489f-aacb-ab327bf3535c None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.236542 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ef5db-3f90-489f-aacb-ab327bf3535c None admin] RBAC: Adding query filter params (name=member) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.236906 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ef5db-3f90-489f-aacb-ab327bf3535c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.238416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ef5db-3f90-489f-aacb-ab327bf3535c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.249873 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 43/86] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.250753 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ef5db-3f90-489f-aacb-ab327bf3535c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.371785 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.391403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a1cba-5e3b-4e64-92e6-5fed9d9bef32 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.421951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ef5db-3f90-489f-aacb-ab327bf3535c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.436084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77187125-3508-4ef6-a92c-5bc30e0a5c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.436897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77187125-3508-4ef6-a92c-5bc30e0a5c1d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.437378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77187125-3508-4ef6-a92c-5bc30e0a5c1d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.437774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77187125-3508-4ef6-a92c-5bc30e0a5c1d None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.438690 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77187125-3508-4ef6-a92c-5bc30e0a5c1d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.439088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77187125-3508-4ef6-a92c-5bc30e0a5c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.440863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77187125-3508-4ef6-a92c-5bc30e0a5c1d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.451996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 44/87] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.452871 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77187125-3508-4ef6-a92c-5bc30e0a5c1d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.483302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a1e14-215c-4ad1-8241-77fcbaab8b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.484734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a1e14-215c-4ad1-8241-77fcbaab8b87 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.485599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a1e14-215c-4ad1-8241-77fcbaab8b87 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.486104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a1e14-215c-4ad1-8241-77fcbaab8b87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.488215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a1e14-215c-4ad1-8241-77fcbaab8b87 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2, user.id=d7063e9478d747e983e7ced895545d44) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.489015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a1e14-215c-4ad1-8241-77fcbaab8b87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.492850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a1e14-215c-4ad1-8241-77fcbaab8b87 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.494089 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.506778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 44/88] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/role_assignments?user.id=d7063e9478d747e983e7ced895545d44&scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.509051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a1e14-215c-4ad1-8241-77fcbaab8b87 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.605690 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77187125-3508-4ef6-a92c-5bc30e0a5c1d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.662895 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29ae89-0d96-45ab-846a-ca33ad95b3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.665266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29ae89-0d96-45ab-846a-ca33ad95b3a2 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.665577 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29ae89-0d96-45ab-846a-ca33ad95b3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.665854 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29ae89-0d96-45ab-846a-ca33ad95b3a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.666648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29ae89-0d96-45ab-846a-ca33ad95b3a2 None admin] RBAC: Adding query filter params (role.id=7ccf7639a4864595b7bc589a9dc44df7, scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2, user.id=dd9b2ddbfdf942fbab0323fe249d8639) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.666957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29ae89-0d96-45ab-846a-ca33ad95b3a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.671150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29ae89-0d96-45ab-846a-ca33ad95b3a2 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.679482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a1e14-215c-4ad1-8241-77fcbaab8b87 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.686729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 45/89] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/role_assignments?user.id=dd9b2ddbfdf942fbab0323fe249d8639&scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2&role.id=7ccf7639a4864595b7bc589a9dc44df7 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.687644 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29ae89-0d96-45ab-846a-ca33ad95b3a2 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.752658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.768816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf502376-11ad-46e8-8e2a-935f6a276efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.769480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf502376-11ad-46e8-8e2a-935f6a276efd None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.769813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf502376-11ad-46e8-8e2a-935f6a276efd None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.770091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf502376-11ad-46e8-8e2a-935f6a276efd None admin] PATH_INFO: `/v3/users/dd9b2ddbfdf942fbab0323fe249d8639` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.775080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf502376-11ad-46e8-8e2a-935f6a276efd None admin] RBAC: Authorizing `identity:get_user(user_id=dd9b2ddbfdf942fbab0323fe249d8639)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.776673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf502376-11ad-46e8-8e2a-935f6a276efd None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.778890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 45/90] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/users/dd9b2ddbfdf942fbab0323fe249d8639 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.786287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf502376-11ad-46e8-8e2a-935f6a276efd None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.911372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.924324 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29ae89-0d96-45ab-846a-ca33ad95b3a2 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.984718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf502376-11ad-46e8-8e2a-935f6a276efd None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.005570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23f5e5-c1c5-4499-ac57-434b7ea67761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.006298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23f5e5-c1c5-4499-ac57-434b7ea67761 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.006582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23f5e5-c1c5-4499-ac57-434b7ea67761 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.006857 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23f5e5-c1c5-4499-ac57-434b7ea67761 None admin] PATH_INFO: `/v3/users/dd9b2ddbfdf942fbab0323fe249d8639` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.008216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23f5e5-c1c5-4499-ac57-434b7ea67761 None admin] RBAC: Authorizing `identity:get_user(user_id=dd9b2ddbfdf942fbab0323fe249d8639)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.009703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23f5e5-c1c5-4499-ac57-434b7ea67761 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.011549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 46/91] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/users/dd9b2ddbfdf942fbab0323fe249d8639 => generated 275 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:03.019952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23f5e5-c1c5-4499-ac57-434b7ea67761 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.073965 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940e1f0-8fc6-4037-93e9-a0664d85747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.075205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940e1f0-8fc6-4037-93e9-a0664d85747b None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.078211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940e1f0-8fc6-4037-93e9-a0664d85747b None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.078554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940e1f0-8fc6-4037-93e9-a0664d85747b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.232530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:03.237406 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940e1f0-8fc6-4037-93e9-a0664d85747b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.238958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940e1f0-8fc6-4037-93e9-a0664d85747b None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.267110 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23f5e5-c1c5-4499-ac57-434b7ea67761 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.321181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c44867-42b4-439f-b8ee-cd850979d867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.322244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c44867-42b4-439f-b8ee-cd850979d867 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.322792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c44867-42b4-439f-b8ee-cd850979d867 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.323320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c44867-42b4-439f-b8ee-cd850979d867 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.394769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:03.471325 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c44867-42b4-439f-b8ee-cd850979d867 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.473522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c44867-42b4-439f-b8ee-cd850979d867 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.525442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 46/92] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:34:03.532534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940e1f0-8fc6-4037-93e9-a0664d85747b None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.718663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 47/93] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:34:03.725198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c44867-42b4-439f-b8ee-cd850979d867 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.783367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:03.789889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940e1f0-8fc6-4037-93e9-a0664d85747b None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.877954 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69f6e8-f4ea-4b25-894c-1d5c78e53b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.878988 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69f6e8-f4ea-4b25-894c-1d5c78e53b30 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.879513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69f6e8-f4ea-4b25-894c-1d5c78e53b30 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.880016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69f6e8-f4ea-4b25-894c-1d5c78e53b30 None admin] PATH_INFO: `/v3/projects/938d2fbefd8f4b5485d317da3840e3e2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.884114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69f6e8-f4ea-4b25-894c-1d5c78e53b30 None admin] RBAC: Authorizing `identity:get_project(project_id=938d2fbefd8f4b5485d317da3840e3e2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.886654 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69f6e8-f4ea-4b25-894c-1d5c78e53b30 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.889089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 47/94] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/projects/938d2fbefd8f4b5485d317da3840e3e2 => generated 312 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:03.895782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69f6e8-f4ea-4b25-894c-1d5c78e53b30 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.899503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:03.963276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c44867-42b4-439f-b8ee-cd850979d867 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:04.029682 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d85294-49f4-4c7e-8584-2bead9ea55ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:04.030726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d85294-49f4-4c7e-8584-2bead9ea55ca None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:04.031255 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d85294-49f4-4c7e-8584-2bead9ea55ca None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:04.031728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d85294-49f4-4c7e-8584-2bead9ea55ca None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:04.035932 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d85294-49f4-4c7e-8584-2bead9ea55ca None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:04.037698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d85294-49f4-4c7e-8584-2bead9ea55ca None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:04.039663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 48/95] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:04.049628 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d85294-49f4-4c7e-8584-2bead9ea55ca None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:04.109540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69f6e8-f4ea-4b25-894c-1d5c78e53b30 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:04.195861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff66d9a-229c-4945-8db3-5bed6802e9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:04.196492 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff66d9a-229c-4945-8db3-5bed6802e9c2 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:04.196777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff66d9a-229c-4945-8db3-5bed6802e9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:04.197057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff66d9a-229c-4945-8db3-5bed6802e9c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:04.197803 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff66d9a-229c-4945-8db3-5bed6802e9c2 None admin] RBAC: Adding query filter params (role.id=7ccf7639a4864595b7bc589a9dc44df7, scope.project.id=938d2fbefd8f4b5485d317da3840e3e2, user.id=dd9b2ddbfdf942fbab0323fe249d8639) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:04.198120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff66d9a-229c-4945-8db3-5bed6802e9c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:04.199750 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff66d9a-229c-4945-8db3-5bed6802e9c2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:04.214463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d85294-49f4-4c7e-8584-2bead9ea55ca None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:04.216664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 48/96] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/role_assignments?user.id=dd9b2ddbfdf942fbab0323fe249d8639&scope.project.id=938d2fbefd8f4b5485d317da3840e3e2&role.id=7ccf7639a4864595b7bc589a9dc44df7 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:04.285366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2290356-1bbf-4094-b2c8-996fc3a126b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:04.286175 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2290356-1bbf-4094-b2c8-996fc3a126b6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:04.286274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2290356-1bbf-4094-b2c8-996fc3a126b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:04.286537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2290356-1bbf-4094-b2c8-996fc3a126b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:04.287344 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2290356-1bbf-4094-b2c8-996fc3a126b6 None admin] RBAC: Adding query filter params (role.id=83a1e8de33194004a2a672104465c819, scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2, user.id=dd9b2ddbfdf942fbab0323fe249d8639) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:04.287654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2290356-1bbf-4094-b2c8-996fc3a126b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:04.289176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2290356-1bbf-4094-b2c8-996fc3a126b6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:04.300090 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 49/97] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:04 2022] GET /identity/v3/role_assignments?user.id=dd9b2ddbfdf942fbab0323fe249d8639&scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2&role.id=83a1e8de33194004a2a672104465c819 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:04.300736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:04.741265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad320b-ae43-4e8b-b805-543ddf01193c None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:04.741777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad320b-ae43-4e8b-b805-543ddf01193c None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:04.741878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad320b-ae43-4e8b-b805-543ddf01193c None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:04.743654 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 49/98] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:04.752098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a710528-4545-45f3-be4e-714e3de5576d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:04.752403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a710528-4545-45f3-be4e-714e3de5576d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:04.752662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a710528-4545-45f3-be4e-714e3de5576d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:04.777262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-6a710528-4545-45f3-be4e-714e3de5576d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:04.778540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6a710528-4545-45f3-be4e-714e3de5576d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:04.871830 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 50/99] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:05.096500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cbe51-d28b-43bd-a64c-f5f3c875ec9c None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.096816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cbe51-d28b-43bd-a64c-f5f3c875ec9c None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.097063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cbe51-d28b-43bd-a64c-f5f3c875ec9c None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.098764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 50/100] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:05.104622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ce041-6d7c-492c-bed3-ebe8a5d43eb0 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.104909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ce041-6d7c-492c-bed3-ebe8a5d43eb0 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.105197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ce041-6d7c-492c-bed3-ebe8a5d43eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.134523 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-1e7ce041-6d7c-492c-bed3-ebe8a5d43eb0 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:05.135844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1e7ce041-6d7c-492c-bed3-ebe8a5d43eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:05.195027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1209672-e633-4adf-a8cf-81781c98c3f3 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.195421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1209672-e633-4adf-a8cf-81781c98c3f3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.195635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1209672-e633-4adf-a8cf-81781c98c3f3 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.197160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 51/101] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:05.203694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86babcd-f55d-4967-9774-1b1e4fd837a9 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.204009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86babcd-f55d-4967-9774-1b1e4fd837a9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.204255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86babcd-f55d-4967-9774-1b1e4fd837a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.230636 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-d86babcd-f55d-4967-9774-1b1e4fd837a9 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:05.232399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d86babcd-f55d-4967-9774-1b1e4fd837a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:05.266309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 51/102] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:05.288389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:05.348455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 52/103] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:05.421366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ce041-6d7c-492c-bed3-ebe8a5d43eb0 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:05.501664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:05.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ce041-6d7c-492c-bed3-ebe8a5d43eb0 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:05.604681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43fd84-733a-4c60-9fd1-56a50932c0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:05.605248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43fd84-733a-4c60-9fd1-56a50932c0cc None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.606072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43fd84-733a-4c60-9fd1-56a50932c0cc None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.606348 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43fd84-733a-4c60-9fd1-56a50932c0cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.623851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43fd84-733a-4c60-9fd1-56a50932c0cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-6c43fd84-733a-4c60-9fd1-56a50932c0cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:05.628826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:05.629663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:05.630411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 52/104] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:05.636178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86babcd-f55d-4967-9774-1b1e4fd837a9 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:05.761516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342a6aa-57b7-464b-a67f-715139cf8a33 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.762382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342a6aa-57b7-464b-a67f-715139cf8a33 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.762697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342a6aa-57b7-464b-a67f-715139cf8a33 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.764655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 53/105] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:05.770343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ed50c-99e9-41a0-aed0-8b9cf52b0317 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.770625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ed50c-99e9-41a0-aed0-8b9cf52b0317 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.770903 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ed50c-99e9-41a0-aed0-8b9cf52b0317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.790155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86babcd-f55d-4967-9774-1b1e4fd837a9 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:05.802457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f19ed50c-99e9-41a0-aed0-8b9cf52b0317 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:05.803807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f19ed50c-99e9-41a0-aed0-8b9cf52b0317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:05.864510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87598723-76df-48d1-b49a-2488c3b0abdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:05.865056 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87598723-76df-48d1-b49a-2488c3b0abdc None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.865344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87598723-76df-48d1-b49a-2488c3b0abdc None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.865623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87598723-76df-48d1-b49a-2488c3b0abdc None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.866311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87598723-76df-48d1-b49a-2488c3b0abdc None admin] RBAC: Adding query filter params (name=reader) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:05.866618 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87598723-76df-48d1-b49a-2488c3b0abdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:05.868078 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87598723-76df-48d1-b49a-2488c3b0abdc None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:05.897247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 53/106] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:05.898128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87598723-76df-48d1-b49a-2488c3b0abdc None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:05.938547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 54/107] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:05.939521 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ed50c-99e9-41a0-aed0-8b9cf52b0317 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:05.942808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.003139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.035150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87598723-76df-48d1-b49a-2488c3b0abdc None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.067369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1019956-edc6-4cf9-a079-780185e79c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.068034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1019956-edc6-4cf9-a079-780185e79c2f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.068320 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1019956-edc6-4cf9-a079-780185e79c2f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.068602 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1019956-edc6-4cf9-a079-780185e79c2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.080842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1019956-edc6-4cf9-a079-780185e79c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-c1019956-edc6-4cf9-a079-780185e79c2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:06.086586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:06.087476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:06.088230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 54/108] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:06.088839 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1019956-edc6-4cf9-a079-780185e79c2f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.122369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ed50c-99e9-41a0-aed0-8b9cf52b0317 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.195941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.207624 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef82e74-49d1-4711-b23b-78de71477a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.208702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef82e74-49d1-4711-b23b-78de71477a16 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.209202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef82e74-49d1-4711-b23b-78de71477a16 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.209304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1019956-edc6-4cf9-a079-780185e79c2f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.209674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef82e74-49d1-4711-b23b-78de71477a16 None admin] PATH_INFO: `/v3/users/68bf81f77d7d40ea902bc3ef6e3a7db2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.248743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef82e74-49d1-4711-b23b-78de71477a16 None admin] RBAC: Authorizing `identity:get_user(user_id=68bf81f77d7d40ea902bc3ef6e3a7db2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.250304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef82e74-49d1-4711-b23b-78de71477a16 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.252541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 55/109] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:05 2022] GET /identity/v3/users/68bf81f77d7d40ea902bc3ef6e3a7db2 => generated 289 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:06.253317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef82e74-49d1-4711-b23b-78de71477a16 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.278291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c783760-d1b0-4c92-bfd8-0eaf31910e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.279063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c783760-d1b0-4c92-bfd8-0eaf31910e15 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.279506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c783760-d1b0-4c92-bfd8-0eaf31910e15 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.280406 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c783760-d1b0-4c92-bfd8-0eaf31910e15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.282099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c783760-d1b0-4c92-bfd8-0eaf31910e15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.284248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c783760-d1b0-4c92-bfd8-0eaf31910e15 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.286420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 55/110] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:06.287683 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c783760-d1b0-4c92-bfd8-0eaf31910e15 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.375709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef82e74-49d1-4711-b23b-78de71477a16 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.385807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.426501 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c783760-d1b0-4c92-bfd8-0eaf31910e15 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.430707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909595d6-ed1a-46a3-bc0c-e840874f74f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.431395 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909595d6-ed1a-46a3-bc0c-e840874f74f3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.431653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909595d6-ed1a-46a3-bc0c-e840874f74f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.431926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909595d6-ed1a-46a3-bc0c-e840874f74f3 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.432617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909595d6-ed1a-46a3-bc0c-e840874f74f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:06.432923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909595d6-ed1a-46a3-bc0c-e840874f74f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.434416 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909595d6-ed1a-46a3-bc0c-e840874f74f3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.449798 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 56/111] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:06.449798 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909595d6-ed1a-46a3-bc0c-e840874f74f3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.473331 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72991b36-f13a-41f3-99a7-771007f18e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.474341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72991b36-f13a-41f3-99a7-771007f18e04 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.474807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72991b36-f13a-41f3-99a7-771007f18e04 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.475281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72991b36-f13a-41f3-99a7-771007f18e04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.493952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72991b36-f13a-41f3-99a7-771007f18e04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-72991b36-f13a-41f3-99a7-771007f18e04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:06.504449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:06.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:06.508209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:06.508209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:06.508209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:06.508209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:06.508209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:06.508209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:06.508209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:06.508209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:06.508209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:06.508209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:06.508209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:06.508209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:06.511519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 56/112] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:06.513585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72991b36-f13a-41f3-99a7-771007f18e04 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.540616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.630176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909595d6-ed1a-46a3-bc0c-e840874f74f3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.691492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852936b3-3d8b-492a-b8d9-c8469e207209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.692473 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852936b3-3d8b-492a-b8d9-c8469e207209 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.693409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852936b3-3d8b-492a-b8d9-c8469e207209 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.693659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852936b3-3d8b-492a-b8d9-c8469e207209 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.703809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72991b36-f13a-41f3-99a7-771007f18e04 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.759542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.765467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296630f-d6ad-4c44-8982-77ef71f0f1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.767155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296630f-d6ad-4c44-8982-77ef71f0f1fa None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.767900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296630f-d6ad-4c44-8982-77ef71f0f1fa None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.768652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296630f-d6ad-4c44-8982-77ef71f0f1fa None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.770024 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296630f-d6ad-4c44-8982-77ef71f0f1fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.772932 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296630f-d6ad-4c44-8982-77ef71f0f1fa None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.820844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852936b3-3d8b-492a-b8d9-c8469e207209 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.823404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 57/113] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:34:06 2022] POST /identity/v3/projects => generated 302 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:06.824139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852936b3-3d8b-492a-b8d9-c8469e207209 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.826153 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296630f-d6ad-4c44-8982-77ef71f0f1fa None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.962763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.044212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 57/114] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:07.045850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852936b3-3d8b-492a-b8d9-c8469e207209 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.065943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296630f-d6ad-4c44-8982-77ef71f0f1fa None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.134196 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390806e7-3bcc-44df-97b1-4fa026aeda79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.135042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390806e7-3bcc-44df-97b1-4fa026aeda79 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.135497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390806e7-3bcc-44df-97b1-4fa026aeda79 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.135897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390806e7-3bcc-44df-97b1-4fa026aeda79 None admin] PATH_INFO: `/v3/users/dd9b2ddbfdf942fbab0323fe249d8639` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.138313 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390806e7-3bcc-44df-97b1-4fa026aeda79 None admin] RBAC: Authorizing `identity:get_user(user_id=dd9b2ddbfdf942fbab0323fe249d8639)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.140315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390806e7-3bcc-44df-97b1-4fa026aeda79 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.142691 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 58/115] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/users/dd9b2ddbfdf942fbab0323fe249d8639 => generated 275 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:07.143574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390806e7-3bcc-44df-97b1-4fa026aeda79 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.211400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.257450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852936b3-3d8b-492a-b8d9-c8469e207209 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.337763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61725bb5-7dce-4428-b66b-a3a2869fe3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.338403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61725bb5-7dce-4428-b66b-a3a2869fe3b3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.338687 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61725bb5-7dce-4428-b66b-a3a2869fe3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.338965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61725bb5-7dce-4428-b66b-a3a2869fe3b3 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.339681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61725bb5-7dce-4428-b66b-a3a2869fe3b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:07.340575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61725bb5-7dce-4428-b66b-a3a2869fe3b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.342437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61725bb5-7dce-4428-b66b-a3a2869fe3b3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.358443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 58/116] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:07.359104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61725bb5-7dce-4428-b66b-a3a2869fe3b3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.371540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390806e7-3bcc-44df-97b1-4fa026aeda79 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.430228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.440715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf21808-f70c-4243-8c86-21a92a5ae7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.441819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf21808-f70c-4243-8c86-21a92a5ae7ee None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.442328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf21808-f70c-4243-8c86-21a92a5ae7ee None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.442827 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf21808-f70c-4243-8c86-21a92a5ae7ee None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.444967 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf21808-f70c-4243-8c86-21a92a5ae7ee None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.447824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf21808-f70c-4243-8c86-21a92a5ae7ee None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.451271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 59/117] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:07.452734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf21808-f70c-4243-8c86-21a92a5ae7ee None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.514059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61725bb5-7dce-4428-b66b-a3a2869fe3b3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.546488 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.554841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761439d9-10b7-45c0-b72e-70fa02f48488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.555535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761439d9-10b7-45c0-b72e-70fa02f48488 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.555814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761439d9-10b7-45c0-b72e-70fa02f48488 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.556084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761439d9-10b7-45c0-b72e-70fa02f48488 None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.557244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761439d9-10b7-45c0-b72e-70fa02f48488 None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.559335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761439d9-10b7-45c0-b72e-70fa02f48488 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.561228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 59/118] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:07.562219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17aecb-9c55-4413-961b-36db24293529 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.562657 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17aecb-9c55-4413-961b-36db24293529 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.562920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17aecb-9c55-4413-961b-36db24293529 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.564181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 60/119] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:07.569196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17aecb-9c55-4413-961b-36db24293529 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.654268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf21808-f70c-4243-8c86-21a92a5ae7ee None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.688181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17aecb-9c55-4413-961b-36db24293529 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.714750 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8a1aa-6199-4dde-a79e-5b2b01bd5176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.716523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8a1aa-6199-4dde-a79e-5b2b01bd5176 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.716948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8a1aa-6199-4dde-a79e-5b2b01bd5176 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.717546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8a1aa-6199-4dde-a79e-5b2b01bd5176 None admin] PATH_INFO: `/v3/users/d7063e9478d747e983e7ced895545d44` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.719253 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8a1aa-6199-4dde-a79e-5b2b01bd5176 None admin] RBAC: Authorizing `identity:get_user(user_id=d7063e9478d747e983e7ced895545d44)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.721106 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8a1aa-6199-4dde-a79e-5b2b01bd5176 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.729867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 60/120] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/users/d7063e9478d747e983e7ced895545d44 => generated 247 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:07.729867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13285bdb-60ad-4b7f-bbb2-58e2a480dd0b None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.729867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13285bdb-60ad-4b7f-bbb2-58e2a480dd0b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.729867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13285bdb-60ad-4b7f-bbb2-58e2a480dd0b None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.729867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 61/121] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:07.731271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934b24c-d373-4eef-aaa5-716b950c2728 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.731448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934b24c-d373-4eef-aaa5-716b950c2728 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.731865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934b24c-d373-4eef-aaa5-716b950c2728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.736682 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1126dcd-09a3-4d18-84ab-998261c1ceea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.737317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1126dcd-09a3-4d18-84ab-998261c1ceea None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.737594 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1126dcd-09a3-4d18-84ab-998261c1ceea None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.737861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1126dcd-09a3-4d18-84ab-998261c1ceea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.738603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1126dcd-09a3-4d18-84ab-998261c1ceea None admin] RBAC: Adding query filter params (role.id=418096fb75b6406f8cb75452cb7b8edb, scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2, user.id=68bf81f77d7d40ea902bc3ef6e3a7db2) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:07.738908 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1126dcd-09a3-4d18-84ab-998261c1ceea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.740425 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1126dcd-09a3-4d18-84ab-998261c1ceea None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.755174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 61/122] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/role_assignments?user.id=68bf81f77d7d40ea902bc3ef6e3a7db2&scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2&role.id=418096fb75b6406f8cb75452cb7b8edb => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:07.755174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1126dcd-09a3-4d18-84ab-998261c1ceea None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.763351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-6934b24c-d373-4eef-aaa5-716b950c2728 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:07.764805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6934b24c-d373-4eef-aaa5-716b950c2728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:07.842523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.850194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.850825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 62/123] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:07.852451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934b24c-d373-4eef-aaa5-716b950c2728 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.963973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1126dcd-09a3-4d18-84ab-998261c1ceea None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.014249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a833f93-5ad1-470c-af7a-024a1de82859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.015921 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a833f93-5ad1-470c-af7a-024a1de82859 None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.016410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a833f93-5ad1-470c-af7a-024a1de82859 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.016909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a833f93-5ad1-470c-af7a-024a1de82859 None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2/users/dd9b2ddbfdf942fbab0323fe249d8639/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.024301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a833f93-5ad1-470c-af7a-024a1de82859 None admin] RBAC: Authorizing `identity:create_grant(project_id=76ea9635caef4c68bc934ba9f9ee98f2, user_id=dd9b2ddbfdf942fbab0323fe249d8639, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.026162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a833f93-5ad1-470c-af7a-024a1de82859 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.087526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 62/124] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:07 2022] PUT /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2/users/dd9b2ddbfdf942fbab0323fe249d8639/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:08.087526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4064f-b4e0-4b3d-b737-095dd4dbff29 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.087526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4064f-b4e0-4b3d-b737-095dd4dbff29 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.087526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4064f-b4e0-4b3d-b737-095dd4dbff29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.118405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934b24c-d373-4eef-aaa5-716b950c2728 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.141155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-c1a4064f-b4e0-4b3d-b737-095dd4dbff29 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:08.142473 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c1a4064f-b4e0-4b3d-b737-095dd4dbff29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:08.213753 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588872cb-e0ca-4038-b675-ea63e3f40dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.214419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588872cb-e0ca-4038-b675-ea63e3f40dbd None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.214698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588872cb-e0ca-4038-b675-ea63e3f40dbd None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.215724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588872cb-e0ca-4038-b675-ea63e3f40dbd None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.217816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588872cb-e0ca-4038-b675-ea63e3f40dbd None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.220355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588872cb-e0ca-4038-b675-ea63e3f40dbd None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.223892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 63/125] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:08.234997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588872cb-e0ca-4038-b675-ea63e3f40dbd None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.326484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 63/126] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:08.336620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:08.396483 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588872cb-e0ca-4038-b675-ea63e3f40dbd None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.438416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59f5a0-8a4b-4296-a39b-c13108d057c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.439445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59f5a0-8a4b-4296-a39b-c13108d057c4 None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.439889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59f5a0-8a4b-4296-a39b-c13108d057c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.440327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59f5a0-8a4b-4296-a39b-c13108d057c4 None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2/users/d7063e9478d747e983e7ced895545d44/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.442644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59f5a0-8a4b-4296-a39b-c13108d057c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=76ea9635caef4c68bc934ba9f9ee98f2, user_id=d7063e9478d747e983e7ced895545d44, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.444402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59f5a0-8a4b-4296-a39b-c13108d057c4 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.473488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 64/127] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:08 2022] PUT /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2/users/d7063e9478d747e983e7ced895545d44/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:34:08.523618 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4064f-b4e0-4b3d-b737-095dd4dbff29 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.597477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:08.661506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4064f-b4e0-4b3d-b737-095dd4dbff29 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.713883 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a979f-9763-4a27-8a0b-23be9d3c2ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.714609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a979f-9763-4a27-8a0b-23be9d3c2ed0 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.715020 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a979f-9763-4a27-8a0b-23be9d3c2ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.715323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a979f-9763-4a27-8a0b-23be9d3c2ed0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.733388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a979f-9763-4a27-8a0b-23be9d3c2ed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-3b8a979f-9763-4a27-8a0b-23be9d3c2ed0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:08.738375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:08.739394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:08.740176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 64/128] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:08.747480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59f5a0-8a4b-4296-a39b-c13108d057c4 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.834241 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a979f-9763-4a27-8a0b-23be9d3c2ed0 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.920762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59f5a0-8a4b-4296-a39b-c13108d057c4 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.993885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:09.017256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e548aa-aae9-4c6c-aee6-6036ad4cb976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.018735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e548aa-aae9-4c6c-aee6-6036ad4cb976 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.023643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e548aa-aae9-4c6c-aee6-6036ad4cb976 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.024480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e548aa-aae9-4c6c-aee6-6036ad4cb976 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.026019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e548aa-aae9-4c6c-aee6-6036ad4cb976 None admin] RBAC: Adding query filter params (name=member) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:09.026863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e548aa-aae9-4c6c-aee6-6036ad4cb976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.030515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e548aa-aae9-4c6c-aee6-6036ad4cb976 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.055304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 65/129] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:09.065624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e548aa-aae9-4c6c-aee6-6036ad4cb976 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.171333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a979f-9763-4a27-8a0b-23be9d3c2ed0 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.218509 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:09.304349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d4fec-cdad-4b70-8375-70d88566ecea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.305369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d4fec-cdad-4b70-8375-70d88566ecea None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.305849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d4fec-cdad-4b70-8375-70d88566ecea None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.306325 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d4fec-cdad-4b70-8375-70d88566ecea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.352750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d4fec-cdad-4b70-8375-70d88566ecea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-ec6d4fec-cdad-4b70-8375-70d88566ecea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:09.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:09.372917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:09.374391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:09.374391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:09.374391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:09.374391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:09.374391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:09.374391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:09.374391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:09.374391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:09.374391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:09.374391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:09.374391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:09.374391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:09.375534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e548aa-aae9-4c6c-aee6-6036ad4cb976 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.375669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 65/130] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 540 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:09.389198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d4fec-cdad-4b70-8375-70d88566ecea None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.450283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df05fd9-7944-49cd-83aa-0c6a9de781f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.451242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df05fd9-7944-49cd-83aa-0c6a9de781f3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.451523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df05fd9-7944-49cd-83aa-0c6a9de781f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.451799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df05fd9-7944-49cd-83aa-0c6a9de781f3 None admin] PATH_INFO: `/v3/users/dd9b2ddbfdf942fbab0323fe249d8639` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.454569 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df05fd9-7944-49cd-83aa-0c6a9de781f3 None admin] RBAC: Authorizing `identity:get_user(user_id=dd9b2ddbfdf942fbab0323fe249d8639)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.456122 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df05fd9-7944-49cd-83aa-0c6a9de781f3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.459846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 66/131] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:09 2022] GET /identity/v3/users/dd9b2ddbfdf942fbab0323fe249d8639 => generated 275 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:09.464927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df05fd9-7944-49cd-83aa-0c6a9de781f3 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.576547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:09.677527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df05fd9-7944-49cd-83aa-0c6a9de781f3 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.732419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d4fec-cdad-4b70-8375-70d88566ecea None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.754895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08545154-639c-4f3b-bff2-e6c925d0c500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.755663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08545154-639c-4f3b-bff2-e6c925d0c500 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.755948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08545154-639c-4f3b-bff2-e6c925d0c500 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.756225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08545154-639c-4f3b-bff2-e6c925d0c500 None admin] PATH_INFO: `/v3/projects/938d2fbefd8f4b5485d317da3840e3e2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.757682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08545154-639c-4f3b-bff2-e6c925d0c500 None admin] RBAC: Authorizing `identity:get_project(project_id=938d2fbefd8f4b5485d317da3840e3e2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.759303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08545154-639c-4f3b-bff2-e6c925d0c500 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.762900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 67/132] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:09 2022] GET /identity/v3/projects/938d2fbefd8f4b5485d317da3840e3e2 => generated 312 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:09.768252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08545154-639c-4f3b-bff2-e6c925d0c500 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.824040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1135-c423-4f96-817d-96d64815d3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.825746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1135-c423-4f96-817d-96d64815d3d0 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.826209 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1135-c423-4f96-817d-96d64815d3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.826651 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1135-c423-4f96-817d-96d64815d3d0 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.827904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d1135-c423-4f96-817d-96d64815d3d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:09.828412 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d1135-c423-4f96-817d-96d64815d3d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.830949 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d1135-c423-4f96-817d-96d64815d3d0 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.871882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 66/133] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:09.880095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1135-c423-4f96-817d-96d64815d3d0 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.023746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:10.042463 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08545154-639c-4f3b-bff2-e6c925d0c500 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.079864 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:10.098083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791701a-bf05-4086-a2ea-487fb96068ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.098767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791701a-bf05-4086-a2ea-487fb96068ec None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.100698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791701a-bf05-4086-a2ea-487fb96068ec None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.100988 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791701a-bf05-4086-a2ea-487fb96068ec None admin] PATH_INFO: `/v3/projects/938d2fbefd8f4b5485d317da3840e3e2/users/dd9b2ddbfdf942fbab0323fe249d8639/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.105399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791701a-bf05-4086-a2ea-487fb96068ec None admin] RBAC: Authorizing `identity:create_grant(project_id=938d2fbefd8f4b5485d317da3840e3e2, user_id=dd9b2ddbfdf942fbab0323fe249d8639, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.106880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791701a-bf05-4086-a2ea-487fb96068ec None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.148794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 68/134] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:09 2022] PUT /identity/v3/projects/938d2fbefd8f4b5485d317da3840e3e2/users/dd9b2ddbfdf942fbab0323fe249d8639/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:34:10.151439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe61bc0-e660-4568-8a23-6944872fff45 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.151790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe61bc0-e660-4568-8a23-6944872fff45 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.152095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe61bc0-e660-4568-8a23-6944872fff45 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.159712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 69/135] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:10.169860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7d6e1-ecf9-47af-88d0-ff5b042a55b7 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.170390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7d6e1-ecf9-47af-88d0-ff5b042a55b7 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.170806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7d6e1-ecf9-47af-88d0-ff5b042a55b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.208383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-60d7d6e1-ecf9-47af-88d0-ff5b042a55b7 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:10.211036 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-60d7d6e1-ecf9-47af-88d0-ff5b042a55b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:10.216034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1135-c423-4f96-817d-96d64815d3d0 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.292376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5d830-708a-464d-b2e3-eabf3a6b933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.293773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5d830-708a-464d-b2e3-eabf3a6b933d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.294036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5d830-708a-464d-b2e3-eabf3a6b933d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.294418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5d830-708a-464d-b2e3-eabf3a6b933d None admin] PATH_INFO: `/v3/users/dd9b2ddbfdf942fbab0323fe249d8639` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.296846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5d830-708a-464d-b2e3-eabf3a6b933d None admin] RBAC: Authorizing `identity:get_user(user_id=dd9b2ddbfdf942fbab0323fe249d8639)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.298094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5d830-708a-464d-b2e3-eabf3a6b933d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.306259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 67/136] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:09 2022] GET /identity/v3/users/dd9b2ddbfdf942fbab0323fe249d8639 => generated 275 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:10.313234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5d830-708a-464d-b2e3-eabf3a6b933d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.367426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 70/137] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:10.486877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5d830-708a-464d-b2e3-eabf3a6b933d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.488961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:10.529891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14250ff2-256b-4320-a8bc-c9c04d8accc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.529891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14250ff2-256b-4320-a8bc-c9c04d8accc3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.529891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14250ff2-256b-4320-a8bc-c9c04d8accc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.530281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14250ff2-256b-4320-a8bc-c9c04d8accc3 None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.531392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14250ff2-256b-4320-a8bc-c9c04d8accc3 None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.532877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14250ff2-256b-4320-a8bc-c9c04d8accc3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.534813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 68/138] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:10 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:10.540596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7d6e1-ecf9-47af-88d0-ff5b042a55b7 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.691625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7d6e1-ecf9-47af-88d0-ff5b042a55b7 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.724099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:10.765399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a470ee-c940-4894-a4d2-a26c7da64950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.766363 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a470ee-c940-4894-a4d2-a26c7da64950 None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.766652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a470ee-c940-4894-a4d2-a26c7da64950 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.766935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a470ee-c940-4894-a4d2-a26c7da64950 None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2/users/dd9b2ddbfdf942fbab0323fe249d8639/roles/83a1e8de33194004a2a672104465c819` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.769228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a470ee-c940-4894-a4d2-a26c7da64950 None admin] RBAC: Authorizing `identity:create_grant(project_id=76ea9635caef4c68bc934ba9f9ee98f2, user_id=dd9b2ddbfdf942fbab0323fe249d8639, role_id=83a1e8de33194004a2a672104465c819)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.770712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a470ee-c940-4894-a4d2-a26c7da64950 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.794781 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14250ff2-256b-4320-a8bc-c9c04d8accc3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.802508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 71/139] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:10 2022] PUT /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2/users/dd9b2ddbfdf942fbab0323fe249d8639/roles/83a1e8de33194004a2a672104465c819 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:10.929364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14250ff2-256b-4320-a8bc-c9c04d8accc3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.971803 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5458e3-4249-4a7f-9856-95c98530e4d7 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.972126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5458e3-4249-4a7f-9856-95c98530e4d7 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.972293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5458e3-4249-4a7f-9856-95c98530e4d7 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.973898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 72/140] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:10.979550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378105-986b-4a11-88fc-2c1ac0bbf996 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.979883 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378105-986b-4a11-88fc-2c1ac0bbf996 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.980170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378105-986b-4a11-88fc-2c1ac0bbf996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.982941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8e861-d232-49ff-970e-3e0542748af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.983748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8e861-d232-49ff-970e-3e0542748af2 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.984128 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8e861-d232-49ff-970e-3e0542748af2 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.984426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8e861-d232-49ff-970e-3e0542748af2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.985709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8e861-d232-49ff-970e-3e0542748af2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.987323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8e861-d232-49ff-970e-3e0542748af2 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.988969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 69/141] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:10.995705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8e861-d232-49ff-970e-3e0542748af2 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:11.002968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-96378105-986b-4a11-88fc-2c1ac0bbf996 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:11.004598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-96378105-986b-4a11-88fc-2c1ac0bbf996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:11.073854 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:11.150664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 73/142] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:11.200780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8e861-d232-49ff-970e-3e0542748af2 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:11.224337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc8d5a-4a99-4960-92cd-e8becd1c209d None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.224574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc8d5a-4a99-4960-92cd-e8becd1c209d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.225116 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc8d5a-4a99-4960-92cd-e8becd1c209d None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.226781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 74/143] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:11.233767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdeb12-0146-4f9c-884b-1c0476d6647b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.234150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdeb12-0146-4f9c-884b-1c0476d6647b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.234497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdeb12-0146-4f9c-884b-1c0476d6647b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.254250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-1dcdeb12-0146-4f9c-884b-1c0476d6647b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:11.255754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1dcdeb12-0146-4f9c-884b-1c0476d6647b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:11.278621 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1d69a-8de4-4270-9094-08bf8f7b8a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:11.280203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1d69a-8de4-4270-9094-08bf8f7b8a85 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.280714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1d69a-8de4-4270-9094-08bf8f7b8a85 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.281212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1d69a-8de4-4270-9094-08bf8f7b8a85 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.282478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1d69a-8de4-4270-9094-08bf8f7b8a85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:11.286572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1d69a-8de4-4270-9094-08bf8f7b8a85 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:11.337232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 70/144] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:10 2022] POST /identity/v3/users => generated 283 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:11.364482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 75/145] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:11.489110 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1d69a-8de4-4270-9094-08bf8f7b8a85 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:11.627328 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:11.638796 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1d69a-8de4-4270-9094-08bf8f7b8a85 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:11.683185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bd582-7a62-487a-8b54-c5017b6a2686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:11.684210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bd582-7a62-487a-8b54-c5017b6a2686 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.684713 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bd582-7a62-487a-8b54-c5017b6a2686 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.685197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bd582-7a62-487a-8b54-c5017b6a2686 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.702149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bd582-7a62-487a-8b54-c5017b6a2686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-276bd582-7a62-487a-8b54-c5017b6a2686 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:11.708197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:11.709133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:11.709931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:11.709931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:11.709931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:11.709931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:11.709931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:11.709931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:11.709931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:11.709931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:11.709931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:11.709931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:11.709931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:11.709931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:11.712569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 71/146] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:11.719463 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdeb12-0146-4f9c-884b-1c0476d6647b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:11.764594 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:11.853520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdeb12-0146-4f9c-884b-1c0476d6647b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:11.899835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bea4e-b51a-4596-ae54-242edc11fc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:11.900453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bea4e-b51a-4596-ae54-242edc11fc01 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.900732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bea4e-b51a-4596-ae54-242edc11fc01 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.901001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bea4e-b51a-4596-ae54-242edc11fc01 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.901672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bea4e-b51a-4596-ae54-242edc11fc01 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:11.901971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bea4e-b51a-4596-ae54-242edc11fc01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:11.903999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bea4e-b51a-4596-ae54-242edc11fc01 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:11.920128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 76/147] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:11.926415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bd582-7a62-487a-8b54-c5017b6a2686 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.022995 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bea4e-b51a-4596-ae54-242edc11fc01 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.111903 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bd582-7a62-487a-8b54-c5017b6a2686 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.158684 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeeae2e-6cbf-4229-98c1-83abcf482c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.159770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeeae2e-6cbf-4229-98c1-83abcf482c72 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.160283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeeae2e-6cbf-4229-98c1-83abcf482c72 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.160874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeeae2e-6cbf-4229-98c1-83abcf482c72 None admin] PATH_INFO: `/v3/users/68bf81f77d7d40ea902bc3ef6e3a7db2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.164714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeeae2e-6cbf-4229-98c1-83abcf482c72 None admin] RBAC: Authorizing `identity:get_user(user_id=68bf81f77d7d40ea902bc3ef6e3a7db2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:12.166646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeeae2e-6cbf-4229-98c1-83abcf482c72 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:12.169612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 72/148] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:11 2022] GET /identity/v3/users/68bf81f77d7d40ea902bc3ef6e3a7db2 => generated 289 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:12.175997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeeae2e-6cbf-4229-98c1-83abcf482c72 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.212359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:12.237058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:12.247504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bea4e-b51a-4596-ae54-242edc11fc01 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.307045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e13d24-0b8c-4589-b599-3938010112fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.307634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e13d24-0b8c-4589-b599-3938010112fa None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.307908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e13d24-0b8c-4589-b599-3938010112fa None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.308174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e13d24-0b8c-4589-b599-3938010112fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.309643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeeae2e-6cbf-4229-98c1-83abcf482c72 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.328899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e13d24-0b8c-4589-b599-3938010112fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-09e13d24-0b8c-4589-b599-3938010112fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:12.334476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:12.335424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:12.336194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 77/149] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:12.348191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e13d24-0b8c-4589-b599-3938010112fa None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.352811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc615f-70a9-430a-8cad-6685f15dc58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.353869 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc615f-70a9-430a-8cad-6685f15dc58b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.354372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc615f-70a9-430a-8cad-6685f15dc58b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.354877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc615f-70a9-430a-8cad-6685f15dc58b None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.356547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc615f-70a9-430a-8cad-6685f15dc58b None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:12.358566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc615f-70a9-430a-8cad-6685f15dc58b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:12.361555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 73/150] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:12 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:12.367517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc615f-70a9-430a-8cad-6685f15dc58b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.478213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:12.491178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e13d24-0b8c-4589-b599-3938010112fa None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.493086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc615f-70a9-430a-8cad-6685f15dc58b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.529818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bd4ed-6852-4160-a294-5b6211cb5ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.530326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bd4ed-6852-4160-a294-5b6211cb5ac7 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.530607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bd4ed-6852-4160-a294-5b6211cb5ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.530880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bd4ed-6852-4160-a294-5b6211cb5ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.531698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bd4ed-6852-4160-a294-5b6211cb5ac7 None admin] RBAC: Adding query filter params (name=member) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:12.531917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bd4ed-6852-4160-a294-5b6211cb5ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:12.533317 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bd4ed-6852-4160-a294-5b6211cb5ac7 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:12.536826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e808c-b132-48f9-a7cf-9d6065d52cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.537900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e808c-b132-48f9-a7cf-9d6065d52cbc None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.538424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e808c-b132-48f9-a7cf-9d6065d52cbc None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.538998 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e808c-b132-48f9-a7cf-9d6065d52cbc None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2/users/68bf81f77d7d40ea902bc3ef6e3a7db2/roles/418096fb75b6406f8cb75452cb7b8edb` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.541402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e808c-b132-48f9-a7cf-9d6065d52cbc None admin] RBAC: Authorizing `identity:create_grant(project_id=76ea9635caef4c68bc934ba9f9ee98f2, user_id=68bf81f77d7d40ea902bc3ef6e3a7db2, role_id=418096fb75b6406f8cb75452cb7b8edb)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:12.542457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 78/151] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:12.544117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e808c-b132-48f9-a7cf-9d6065d52cbc None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:12.549075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bd4ed-6852-4160-a294-5b6211cb5ac7 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.563934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 74/152] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:12 2022] PUT /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2/users/68bf81f77d7d40ea902bc3ef6e3a7db2/roles/418096fb75b6406f8cb75452cb7b8edb => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:34:12.573376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:12.675412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bd4ed-6852-4160-a294-5b6211cb5ac7 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.716224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12d33c-d2b5-4926-9802-4df2ae3c1434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.716967 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12d33c-d2b5-4926-9802-4df2ae3c1434 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.717243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12d33c-d2b5-4926-9802-4df2ae3c1434 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.717509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12d33c-d2b5-4926-9802-4df2ae3c1434 None admin] PATH_INFO: `/v3/users/dd9b2ddbfdf942fbab0323fe249d8639` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.718827 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12d33c-d2b5-4926-9802-4df2ae3c1434 None admin] RBAC: Authorizing `identity:get_user(user_id=dd9b2ddbfdf942fbab0323fe249d8639)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:12.720399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12d33c-d2b5-4926-9802-4df2ae3c1434 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:12.722159 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 79/153] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:12 2022] GET /identity/v3/users/dd9b2ddbfdf942fbab0323fe249d8639 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:12.728718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e808c-b132-48f9-a7cf-9d6065d52cbc None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.922417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e808c-b132-48f9-a7cf-9d6065d52cbc None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.012482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44316d-8b15-4d40-ad0b-4682019d3bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.013541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44316d-8b15-4d40-ad0b-4682019d3bc3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.014037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44316d-8b15-4d40-ad0b-4682019d3bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.014510 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44316d-8b15-4d40-ad0b-4682019d3bc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.056916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.165155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44316d-8b15-4d40-ad0b-4682019d3bc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.168521 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44316d-8b15-4d40-ad0b-4682019d3bc3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.205098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a4e25-8519-4d09-8bd4-15abc982f483 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.205780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a4e25-8519-4d09-8bd4-15abc982f483 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.206290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a4e25-8519-4d09-8bd4-15abc982f483 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.211145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 80/154] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:13.219500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e530ea-6fbe-49f6-ba59-642a7ddafa04 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.220112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e530ea-6fbe-49f6-ba59-642a7ddafa04 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.220603 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e530ea-6fbe-49f6-ba59-642a7ddafa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.254601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-54e530ea-6fbe-49f6-ba59-642a7ddafa04 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:13.256669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-54e530ea-6fbe-49f6-ba59-642a7ddafa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:13.370104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 81/155] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:13.393695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 75/156] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:34:13.403069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e530ea-6fbe-49f6-ba59-642a7ddafa04 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.423538 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.542478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e530ea-6fbe-49f6-ba59-642a7ddafa04 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.586775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71c25c-8371-4468-a771-142ce8e6973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.587471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71c25c-8371-4468-a771-142ce8e6973f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.587749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71c25c-8371-4468-a771-142ce8e6973f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.588020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71c25c-8371-4468-a771-142ce8e6973f None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.589244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71c25c-8371-4468-a771-142ce8e6973f None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.591387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71c25c-8371-4468-a771-142ce8e6973f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.594570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 82/157] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:13.603483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44316d-8b15-4d40-ad0b-4682019d3bc3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.727076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.834604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44316d-8b15-4d40-ad0b-4682019d3bc3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.908047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5d95e-fb0d-448e-bc07-ab2c07f52e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.909100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5d95e-fb0d-448e-bc07-ab2c07f52e45 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.909590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5d95e-fb0d-448e-bc07-ab2c07f52e45 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.910077 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5d95e-fb0d-448e-bc07-ab2c07f52e45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.911464 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5d95e-fb0d-448e-bc07-ab2c07f52e45 None admin] RBAC: Adding query filter params (role.id=7ccf7639a4864595b7bc589a9dc44df7, scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2, user.id=dd9b2ddbfdf942fbab0323fe249d8639) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:13.912021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5d95e-fb0d-448e-bc07-ab2c07f52e45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.914755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5d95e-fb0d-448e-bc07-ab2c07f52e45 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.927929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 76/158] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/role_assignments?user.id=dd9b2ddbfdf942fbab0323fe249d8639&scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2&role.id=7ccf7639a4864595b7bc589a9dc44df7 => generated 608 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:14.268783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71c25c-8371-4468-a771-142ce8e6973f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:14.395507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:14.420291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71c25c-8371-4468-a771-142ce8e6973f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:14.463554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74ffe3-38aa-4098-92ae-bbaa5c6502fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:14.464208 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74ffe3-38aa-4098-92ae-bbaa5c6502fd None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.464486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74ffe3-38aa-4098-92ae-bbaa5c6502fd None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.465413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74ffe3-38aa-4098-92ae-bbaa5c6502fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.482545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74ffe3-38aa-4098-92ae-bbaa5c6502fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-3f74ffe3-38aa-4098-92ae-bbaa5c6502fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:14.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:14.488911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:14.489672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 83/159] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:14.500297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5d95e-fb0d-448e-bc07-ab2c07f52e45 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:14.651706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5d95e-fb0d-448e-bc07-ab2c07f52e45 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:14.667549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:14.691527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4ad45-6f1e-4306-96ed-6c50151914b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:14.692155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4ad45-6f1e-4306-96ed-6c50151914b1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.692423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4ad45-6f1e-4306-96ed-6c50151914b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.692697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4ad45-6f1e-4306-96ed-6c50151914b1 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.693372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4ad45-6f1e-4306-96ed-6c50151914b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:14.693681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4ad45-6f1e-4306-96ed-6c50151914b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:14.695103 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4ad45-6f1e-4306-96ed-6c50151914b1 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:14.705123 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 77/160] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:14.713976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74ffe3-38aa-4098-92ae-bbaa5c6502fd None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:14.896045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74ffe3-38aa-4098-92ae-bbaa5c6502fd None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:14.955983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a54fa5-f883-4e8d-a90f-3759b45d96c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:14.956654 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a54fa5-f883-4e8d-a90f-3759b45d96c3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.956930 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a54fa5-f883-4e8d-a90f-3759b45d96c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.957203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a54fa5-f883-4e8d-a90f-3759b45d96c3 None admin] PATH_INFO: `/v3/users/d7063e9478d747e983e7ced895545d44` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.958484 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a54fa5-f883-4e8d-a90f-3759b45d96c3 None admin] RBAC: Authorizing `identity:get_user(user_id=d7063e9478d747e983e7ced895545d44)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:14.959995 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a54fa5-f883-4e8d-a90f-3759b45d96c3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:14.963151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 84/161] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:14 2022] GET /identity/v3/users/d7063e9478d747e983e7ced895545d44 => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:14.976288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4ad45-6f1e-4306-96ed-6c50151914b1 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.138358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4ad45-6f1e-4306-96ed-6c50151914b1 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:15.217829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9ec16-9178-47b3-b2ec-bd1a90395604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:15.218766 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ec16-9178-47b3-b2ec-bd1a90395604 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.219280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ec16-9178-47b3-b2ec-bd1a90395604 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.219769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ec16-9178-47b3-b2ec-bd1a90395604 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.282033 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:15.317463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9ec16-9178-47b3-b2ec-bd1a90395604 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:15.318944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9ec16-9178-47b3-b2ec-bd1a90395604 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:15.513097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 78/162] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:15.521698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a54fa5-f883-4e8d-a90f-3759b45d96c3 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.548174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:15.694681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a54fa5-f883-4e8d-a90f-3759b45d96c3 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:15.733381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5450c-db2b-4fb0-969e-b93e926965d1 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.733701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5450c-db2b-4fb0-969e-b93e926965d1 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.733934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5450c-db2b-4fb0-969e-b93e926965d1 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.739699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 79/163] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:15.752782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e774b8-4e55-426b-9ba8-0a0e6c2677f6 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.752962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ad322-a6c5-48b8-8fe3-f3c392c4942d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:15.753079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e774b8-4e55-426b-9ba8-0a0e6c2677f6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.753315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e774b8-4e55-426b-9ba8-0a0e6c2677f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.753489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ad322-a6c5-48b8-8fe3-f3c392c4942d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.753738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ad322-a6c5-48b8-8fe3-f3c392c4942d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.754015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ad322-a6c5-48b8-8fe3-f3c392c4942d None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.755247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ad322-a6c5-48b8-8fe3-f3c392c4942d None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:15.756698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ad322-a6c5-48b8-8fe3-f3c392c4942d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:15.758381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 85/164] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:15.764436 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ad322-a6c5-48b8-8fe3-f3c392c4942d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.784224 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b0e774b8-4e55-426b-9ba8-0a0e6c2677f6 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:15.785454 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b0e774b8-4e55-426b-9ba8-0a0e6c2677f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:15.891104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:15.912773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 80/165] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:15.914003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a155d86-f7c1-48b0-973d-ee117af68dbb None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.914291 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a155d86-f7c1-48b0-973d-ee117af68dbb None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.914558 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a155d86-f7c1-48b0-973d-ee117af68dbb None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.915898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 81/166] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:15.923468 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ad322-a6c5-48b8-8fe3-f3c392c4942d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:15.944686 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bd112-7167-44de-8c50-6aa952402387 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.944992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bd112-7167-44de-8c50-6aa952402387 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.945238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bd112-7167-44de-8c50-6aa952402387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.963628 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:15.980640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-202bd112-7167-44de-8c50-6aa952402387 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:15.981889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-202bd112-7167-44de-8c50-6aa952402387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:15.992895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af03f5-9ff0-43e2-bf35-af6d4d36f870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:15.993523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af03f5-9ff0-43e2-bf35-af6d4d36f870 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.993802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af03f5-9ff0-43e2-bf35-af6d4d36f870 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.994069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af03f5-9ff0-43e2-bf35-af6d4d36f870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.994809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af03f5-9ff0-43e2-bf35-af6d4d36f870 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2, user.id=d7063e9478d747e983e7ced895545d44) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:15.995143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af03f5-9ff0-43e2-bf35-af6d4d36f870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:15.997156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af03f5-9ff0-43e2-bf35-af6d4d36f870 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.009686 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 86/167] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/role_assignments?user.id=d7063e9478d747e983e7ced895545d44&scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:16.088278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 82/168] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:16.391337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01f4b8-6267-45db-8e0a-a5e12cfbb0a3 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.391682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01f4b8-6267-45db-8e0a-a5e12cfbb0a3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.391783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01f4b8-6267-45db-8e0a-a5e12cfbb0a3 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.393552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 87/169] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:16.399468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef500812-71ab-4d8d-94a5-d0d2037e757b None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.400095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef500812-71ab-4d8d-94a5-d0d2037e757b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.400710 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef500812-71ab-4d8d-94a5-d0d2037e757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.424940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ef500812-71ab-4d8d-94a5-d0d2037e757b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:16.426479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ef500812-71ab-4d8d-94a5-d0d2037e757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:16.520751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 83/170] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:16.598722 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01f4b8-6267-45db-8e0a-a5e12cfbb0a3 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.651351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef500812-71ab-4d8d-94a5-d0d2037e757b None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.757836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01f4b8-6267-45db-8e0a-a5e12cfbb0a3 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.767482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.817316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f20d2-d133-4c23-8863-87a03605f5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.818327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f20d2-d133-4c23-8863-87a03605f5ae None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.819006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f20d2-d133-4c23-8863-87a03605f5ae None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.819480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f20d2-d133-4c23-8863-87a03605f5ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.838124 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef500812-71ab-4d8d-94a5-d0d2037e757b None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.838924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f20d2-d133-4c23-8863-87a03605f5ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-485f20d2-d133-4c23-8863-87a03605f5ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:16.845080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:16.846161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:16.846957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:16.846957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:16.846957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:16.846957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:16.846957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:16.846957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:16.846957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:16.846957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:16.846957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:16.846957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:16.846957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:16.846957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:16.848999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 88/171] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:16.850407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7ca9d-5c52-41b9-b3f1-d8c186936007 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.850845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7ca9d-5c52-41b9-b3f1-d8c186936007 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.851296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7ca9d-5c52-41b9-b3f1-d8c186936007 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.852756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 89/172] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:16.856534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7ca9d-5c52-41b9-b3f1-d8c186936007 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.866329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.912741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8a194-f6e0-48fa-ac45-1d2a7ce65988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.913583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8a194-f6e0-48fa-ac45-1d2a7ce65988 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.914076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8a194-f6e0-48fa-ac45-1d2a7ce65988 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.914545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8a194-f6e0-48fa-ac45-1d2a7ce65988 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.960173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8a194-f6e0-48fa-ac45-1d2a7ce65988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-d2e8a194-f6e0-48fa-ac45-1d2a7ce65988 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:16.965222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:16.966135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:16.966917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:16.966917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:16.966917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:16.966917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:16.966917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:16.966917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:16.966917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:16.966917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:16.966917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:16.966917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:16.966917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:16.966917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:16.971404 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 84/173] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:16.972844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c16d-108e-4bd9-a11d-6d76d31323a8 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.973131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c16d-108e-4bd9-a11d-6d76d31323a8 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.973407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c16d-108e-4bd9-a11d-6d76d31323a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.998329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ec09c16d-108e-4bd9-a11d-6d76d31323a8 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:16.999602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ec09c16d-108e-4bd9-a11d-6d76d31323a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:17.032304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7ca9d-5c52-41b9-b3f1-d8c186936007 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.077014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 85/174] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:17.077306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19f6a7-3cda-46f8-b2db-faad5b419a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.077863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19f6a7-3cda-46f8-b2db-faad5b419a42 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.079157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19f6a7-3cda-46f8-b2db-faad5b419a42 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.079361 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c16d-108e-4bd9-a11d-6d76d31323a8 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.079530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19f6a7-3cda-46f8-b2db-faad5b419a42 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.079530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19f6a7-3cda-46f8-b2db-faad5b419a42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.079530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19f6a7-3cda-46f8-b2db-faad5b419a42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.081136 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19f6a7-3cda-46f8-b2db-faad5b419a42 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.097492 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 90/175] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:17.099314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19f6a7-3cda-46f8-b2db-faad5b419a42 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.177457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.248516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.258606 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c16d-108e-4bd9-a11d-6d76d31323a8 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.296469 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527c3a7-db09-4963-9c93-d11a2632ffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.297093 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527c3a7-db09-4963-9c93-d11a2632ffee None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.297366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527c3a7-db09-4963-9c93-d11a2632ffee None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.297635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527c3a7-db09-4963-9c93-d11a2632ffee None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.298306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527c3a7-db09-4963-9c93-d11a2632ffee None admin] RBAC: Adding query filter params (name=member) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.298609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527c3a7-db09-4963-9c93-d11a2632ffee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.298837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19f6a7-3cda-46f8-b2db-faad5b419a42 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.300056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527c3a7-db09-4963-9c93-d11a2632ffee None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.312168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 86/176] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:17.312168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527c3a7-db09-4963-9c93-d11a2632ffee None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.364043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c22a8-8510-477b-86b9-628c12afccda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.364694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c22a8-8510-477b-86b9-628c12afccda None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.364968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c22a8-8510-477b-86b9-628c12afccda None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.365238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c22a8-8510-477b-86b9-628c12afccda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.380188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c22a8-8510-477b-86b9-628c12afccda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-990c22a8-8510-477b-86b9-628c12afccda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:17.385552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:17.386878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:17.388177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 91/177] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:17.388847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce1ee9-ce2b-4fdb-99df-b23c59a130d9 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.388847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce1ee9-ce2b-4fdb-99df-b23c59a130d9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.388847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce1ee9-ce2b-4fdb-99df-b23c59a130d9 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.390176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 92/178] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:17.395495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2113898-3309-46cd-ae6a-21ca7581e779 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.395775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2113898-3309-46cd-ae6a-21ca7581e779 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.396053 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2113898-3309-46cd-ae6a-21ca7581e779 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.397412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 93/179] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:17.398119 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2113898-3309-46cd-ae6a-21ca7581e779 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.448917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527c3a7-db09-4963-9c93-d11a2632ffee None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.468600 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.491422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33968d58-5b64-48d4-9725-611ed1da28fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.492483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33968d58-5b64-48d4-9725-611ed1da28fa None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.492966 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33968d58-5b64-48d4-9725-611ed1da28fa None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.493456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33968d58-5b64-48d4-9725-611ed1da28fa None admin] PATH_INFO: `/v3/users/dd9b2ddbfdf942fbab0323fe249d8639` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.495485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33968d58-5b64-48d4-9725-611ed1da28fa None admin] RBAC: Authorizing `identity:get_user(user_id=dd9b2ddbfdf942fbab0323fe249d8639)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.498078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33968d58-5b64-48d4-9725-611ed1da28fa None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.500949 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 87/180] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/users/dd9b2ddbfdf942fbab0323fe249d8639 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:17.502370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33968d58-5b64-48d4-9725-611ed1da28fa None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.517150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2113898-3309-46cd-ae6a-21ca7581e779 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.546901 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.556298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d928856-6dad-44d2-b563-6db6ba9fda52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.556941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d928856-6dad-44d2-b563-6db6ba9fda52 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.557231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d928856-6dad-44d2-b563-6db6ba9fda52 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.557474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d928856-6dad-44d2-b563-6db6ba9fda52 None admin] PATH_INFO: `/v3/users/dd9b2ddbfdf942fbab0323fe249d8639` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.558761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d928856-6dad-44d2-b563-6db6ba9fda52 None admin] RBAC: Authorizing `identity:get_user(user_id=dd9b2ddbfdf942fbab0323fe249d8639)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.560286 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d928856-6dad-44d2-b563-6db6ba9fda52 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.561927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 94/181] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/users/dd9b2ddbfdf942fbab0323fe249d8639 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:17.562700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d928856-6dad-44d2-b563-6db6ba9fda52 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.633834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33968d58-5b64-48d4-9725-611ed1da28fa None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.682214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675035a4-643e-46bd-9210-8c87880e5720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.682812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d928856-6dad-44d2-b563-6db6ba9fda52 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.683591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675035a4-643e-46bd-9210-8c87880e5720 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.683591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675035a4-643e-46bd-9210-8c87880e5720 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.684097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675035a4-643e-46bd-9210-8c87880e5720 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.700245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675035a4-643e-46bd-9210-8c87880e5720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-675035a4-643e-46bd-9210-8c87880e5720 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:17.706759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:17.708144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:17.709574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 88/182] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:17.713136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481833d-d27b-4961-8575-29c8ca1d97d2 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.713438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481833d-d27b-4961-8575-29c8ca1d97d2 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.713878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481833d-d27b-4961-8575-29c8ca1d97d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.726234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a60551-365b-4bad-a868-9879632d8549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.726859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a60551-365b-4bad-a868-9879632d8549 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.727165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a60551-365b-4bad-a868-9879632d8549 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.727450 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a60551-365b-4bad-a868-9879632d8549 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.728139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a60551-365b-4bad-a868-9879632d8549 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.728442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a60551-365b-4bad-a868-9879632d8549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.729854 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a60551-365b-4bad-a868-9879632d8549 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.739344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 95/183] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:17.740648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569052d9-e376-4417-b46b-95e5246dcbb5 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.740933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569052d9-e376-4417-b46b-95e5246dcbb5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.741203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569052d9-e376-4417-b46b-95e5246dcbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.747415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-2481833d-d27b-4961-8575-29c8ca1d97d2 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:17.748974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2481833d-d27b-4961-8575-29c8ca1d97d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:17.762268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-569052d9-e376-4417-b46b-95e5246dcbb5 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:17.763854 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-569052d9-e376-4417-b46b-95e5246dcbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:17.817158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.827144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.853836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 96/184] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:17.854708 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569052d9-e376-4417-b46b-95e5246dcbb5 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.871279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 89/185] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:17.872398 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481833d-d27b-4961-8575-29c8ca1d97d2 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.008733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569052d9-e376-4417-b46b-95e5246dcbb5 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.039050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481833d-d27b-4961-8575-29c8ca1d97d2 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.065199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d8d62-d8fa-48fd-a40c-4932e9d9e374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.066346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d8d62-d8fa-48fd-a40c-4932e9d9e374 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.066692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d8d62-d8fa-48fd-a40c-4932e9d9e374 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.067103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d8d62-d8fa-48fd-a40c-4932e9d9e374 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.093826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540ee99-e551-4357-9547-29e626e80667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.094900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540ee99-e551-4357-9547-29e626e80667 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.095423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540ee99-e551-4357-9547-29e626e80667 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.095903 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540ee99-e551-4357-9547-29e626e80667 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.150442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.173581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.185065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540ee99-e551-4357-9547-29e626e80667 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.186877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540ee99-e551-4357-9547-29e626e80667 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.200814 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d8d62-d8fa-48fd-a40c-4932e9d9e374 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.203681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d8d62-d8fa-48fd-a40c-4932e9d9e374 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.317541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 90/186] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:18.317541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540ee99-e551-4357-9547-29e626e80667 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.377215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 97/187] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:18.377215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d8d62-d8fa-48fd-a40c-4932e9d9e374 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.428001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.462638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540ee99-e551-4357-9547-29e626e80667 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.493536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.505739 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe53c7a-20d4-4200-9faa-c4c528206f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.506462 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe53c7a-20d4-4200-9faa-c4c528206f13 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.506838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe53c7a-20d4-4200-9faa-c4c528206f13 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.507277 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe53c7a-20d4-4200-9faa-c4c528206f13 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.508051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe53c7a-20d4-4200-9faa-c4c528206f13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:18.508436 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe53c7a-20d4-4200-9faa-c4c528206f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.509944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe53c7a-20d4-4200-9faa-c4c528206f13 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.521939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d8d62-d8fa-48fd-a40c-4932e9d9e374 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.529744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 91/188] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:18.529744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe53c7a-20d4-4200-9faa-c4c528206f13 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.589376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ac105-e1e4-45a8-8802-dbd63d50d6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.590326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ac105-e1e4-45a8-8802-dbd63d50d6d8 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.590814 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ac105-e1e4-45a8-8802-dbd63d50d6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.591281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ac105-e1e4-45a8-8802-dbd63d50d6d8 None admin] PATH_INFO: `/v3/users/6410170d10c74f3ead3234ddbd115d71` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.610815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ac105-e1e4-45a8-8802-dbd63d50d6d8 None admin] RBAC: Authorizing `identity:get_user(user_id=6410170d10c74f3ead3234ddbd115d71)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.612337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ac105-e1e4-45a8-8802-dbd63d50d6d8 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.614076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 98/189] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/users/6410170d10c74f3ead3234ddbd115d71 => generated 283 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:18.614076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ac105-e1e4-45a8-8802-dbd63d50d6d8 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.660981 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe53c7a-20d4-4200-9faa-c4c528206f13 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.711232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98c8cb-c044-4fe0-8df9-fe119aed76f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.712194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98c8cb-c044-4fe0-8df9-fe119aed76f1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.712513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98c8cb-c044-4fe0-8df9-fe119aed76f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.712836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98c8cb-c044-4fe0-8df9-fe119aed76f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.722806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.728207 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98c8cb-c044-4fe0-8df9-fe119aed76f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-1c98c8cb-c044-4fe0-8df9-fe119aed76f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:18.736295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:18.738111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:18.740112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:18.740112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:18.740112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:18.740112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:18.740112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:18.740112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:18.740112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:18.740112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:18.740112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:18.740112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:18.740112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:18.740112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:18.741409 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ac105-e1e4-45a8-8802-dbd63d50d6d8 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.742078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 92/190] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:18.742078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98c8cb-c044-4fe0-8df9-fe119aed76f1 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.779059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379f0db-8608-46a7-b275-5d9fcab1ba5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.779872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379f0db-8608-46a7-b275-5d9fcab1ba5a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.780216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379f0db-8608-46a7-b275-5d9fcab1ba5a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.780563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379f0db-8608-46a7-b275-5d9fcab1ba5a None admin] PATH_INFO: `/v3/projects/938d2fbefd8f4b5485d317da3840e3e2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.782215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379f0db-8608-46a7-b275-5d9fcab1ba5a None admin] RBAC: Authorizing `identity:get_project(project_id=938d2fbefd8f4b5485d317da3840e3e2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.784025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379f0db-8608-46a7-b275-5d9fcab1ba5a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.786214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 99/191] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/projects/938d2fbefd8f4b5485d317da3840e3e2 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:18.792191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379f0db-8608-46a7-b275-5d9fcab1ba5a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.815604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.868609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98c8cb-c044-4fe0-8df9-fe119aed76f1 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.918850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9499bb-4acf-4afa-81ad-103adde11ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.919803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9499bb-4acf-4afa-81ad-103adde11ae6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.920135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9499bb-4acf-4afa-81ad-103adde11ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.920470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9499bb-4acf-4afa-81ad-103adde11ae6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.921939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9499bb-4acf-4afa-81ad-103adde11ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.923647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9499bb-4acf-4afa-81ad-103adde11ae6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.925433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 93/192] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:18.926370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9499bb-4acf-4afa-81ad-103adde11ae6 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.948393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379f0db-8608-46a7-b275-5d9fcab1ba5a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.004114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbdc35-fa3b-442b-86d5-4b811d6cc7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.004881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbdc35-fa3b-442b-86d5-4b811d6cc7dc None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.005269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbdc35-fa3b-442b-86d5-4b811d6cc7dc None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.005674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbdc35-fa3b-442b-86d5-4b811d6cc7dc None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.007366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbdc35-fa3b-442b-86d5-4b811d6cc7dc None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.009288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbdc35-fa3b-442b-86d5-4b811d6cc7dc None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.011115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 100/193] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.012070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbdc35-fa3b-442b-86d5-4b811d6cc7dc None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.031777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.091584 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9499bb-4acf-4afa-81ad-103adde11ae6 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.140158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.156471 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65455676-900c-4b1d-8d44-051d8da8cf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.157446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65455676-900c-4b1d-8d44-051d8da8cf42 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.157868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65455676-900c-4b1d-8d44-051d8da8cf42 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.158344 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65455676-900c-4b1d-8d44-051d8da8cf42 None admin] PATH_INFO: `/v3/users/68bf81f77d7d40ea902bc3ef6e3a7db2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.161152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65455676-900c-4b1d-8d44-051d8da8cf42 None admin] RBAC: Authorizing `identity:get_user(user_id=68bf81f77d7d40ea902bc3ef6e3a7db2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.163743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65455676-900c-4b1d-8d44-051d8da8cf42 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.165797 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbdc35-fa3b-442b-86d5-4b811d6cc7dc None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.172633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 94/194] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/users/68bf81f77d7d40ea902bc3ef6e3a7db2 => generated 289 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.172633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65455676-900c-4b1d-8d44-051d8da8cf42 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.230773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baaa5d6-36a1-4114-ac6d-939cb96481d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.232633 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baaa5d6-36a1-4114-ac6d-939cb96481d2 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.233098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baaa5d6-36a1-4114-ac6d-939cb96481d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.233543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baaa5d6-36a1-4114-ac6d-939cb96481d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.326991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baaa5d6-36a1-4114-ac6d-939cb96481d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.328524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baaa5d6-36a1-4114-ac6d-939cb96481d2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.348471 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65455676-900c-4b1d-8d44-051d8da8cf42 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.389802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b6005-9f34-4d20-872d-e4198e3abddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.390501 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b6005-9f34-4d20-872d-e4198e3abddf None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.390947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b6005-9f34-4d20-872d-e4198e3abddf None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.391318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b6005-9f34-4d20-872d-e4198e3abddf None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.392015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b6005-9f34-4d20-872d-e4198e3abddf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.392314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b6005-9f34-4d20-872d-e4198e3abddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.393747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b6005-9f34-4d20-872d-e4198e3abddf None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.395606 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.409855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 95/195] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.411607 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b6005-9f34-4d20-872d-e4198e3abddf None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.463188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.493061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 101/196] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:19.499007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baaa5d6-36a1-4114-ac6d-939cb96481d2 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.566173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b6005-9f34-4d20-872d-e4198e3abddf None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.624317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e0ee9-2b3f-49dd-a6bf-54d1c3f6a056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.625353 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e0ee9-2b3f-49dd-a6bf-54d1c3f6a056 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.625851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e0ee9-2b3f-49dd-a6bf-54d1c3f6a056 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.626349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e0ee9-2b3f-49dd-a6bf-54d1c3f6a056 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.627853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e0ee9-2b3f-49dd-a6bf-54d1c3f6a056 None admin] RBAC: Adding query filter params (role.id=7ccf7639a4864595b7bc589a9dc44df7, scope.project.id=938d2fbefd8f4b5485d317da3840e3e2, user.id=dd9b2ddbfdf942fbab0323fe249d8639) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.628417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e0ee9-2b3f-49dd-a6bf-54d1c3f6a056 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.630650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baaa5d6-36a1-4114-ac6d-939cb96481d2 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.631293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e0ee9-2b3f-49dd-a6bf-54d1c3f6a056 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.640229 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 96/197] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/role_assignments?user.id=dd9b2ddbfdf942fbab0323fe249d8639&scope.project.id=938d2fbefd8f4b5485d317da3840e3e2&role.id=7ccf7639a4864595b7bc589a9dc44df7 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.641420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e0ee9-2b3f-49dd-a6bf-54d1c3f6a056 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.679071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd71ac0-46d0-48af-b44e-4bb3c25a4b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.680019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd71ac0-46d0-48af-b44e-4bb3c25a4b93 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.680380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd71ac0-46d0-48af-b44e-4bb3c25a4b93 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.680779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd71ac0-46d0-48af-b44e-4bb3c25a4b93 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.681830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd71ac0-46d0-48af-b44e-4bb3c25a4b93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.683789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd71ac0-46d0-48af-b44e-4bb3c25a4b93 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.711765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 102/198] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:19 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:19.712438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd71ac0-46d0-48af-b44e-4bb3c25a4b93 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.751927 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.781502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.869616 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e0ee9-2b3f-49dd-a6bf-54d1c3f6a056 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.890986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd71ac0-46d0-48af-b44e-4bb3c25a4b93 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.939317 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4827f-40a8-4360-9b3c-af96836ff03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.939959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4827f-40a8-4360-9b3c-af96836ff03a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.940237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4827f-40a8-4360-9b3c-af96836ff03a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.940509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4827f-40a8-4360-9b3c-af96836ff03a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.956416 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c6840-f8d3-4e55-93a9-9438db349e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.957225 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c6840-f8d3-4e55-93a9-9438db349e80 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.957680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c6840-f8d3-4e55-93a9-9438db349e80 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.958236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c6840-f8d3-4e55-93a9-9438db349e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.960500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c6840-f8d3-4e55-93a9-9438db349e80 None admin] RBAC: Adding query filter params (role.id=83a1e8de33194004a2a672104465c819, scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2, user.id=dd9b2ddbfdf942fbab0323fe249d8639) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.960500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c6840-f8d3-4e55-93a9-9438db349e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.962357 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c6840-f8d3-4e55-93a9-9438db349e80 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.975744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 97/199] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/role_assignments?user.id=dd9b2ddbfdf942fbab0323fe249d8639&scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2&role.id=83a1e8de33194004a2a672104465c819 => generated 608 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.977100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c6840-f8d3-4e55-93a9-9438db349e80 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.081360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4827f-40a8-4360-9b3c-af96836ff03a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.084005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4827f-40a8-4360-9b3c-af96836ff03a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.161044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:20.190884 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c6840-f8d3-4e55-93a9-9438db349e80 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:20.238886 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:20.259597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332caf6-d487-405f-a136-0dd153af6e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:20.260668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332caf6-d487-405f-a136-0dd153af6e40 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.261150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332caf6-d487-405f-a136-0dd153af6e40 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.262335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332caf6-d487-405f-a136-0dd153af6e40 None admin] PATH_INFO: `/v3/users/6410170d10c74f3ead3234ddbd115d71` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.264427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332caf6-d487-405f-a136-0dd153af6e40 None admin] RBAC: Authorizing `identity:get_user(user_id=6410170d10c74f3ead3234ddbd115d71)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.268036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332caf6-d487-405f-a136-0dd153af6e40 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.271343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 98/200] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/users/6410170d10c74f3ead3234ddbd115d71 => generated 283 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:20.272829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332caf6-d487-405f-a136-0dd153af6e40 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.275659 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 103/201] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:20.282114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4827f-40a8-4360-9b3c-af96836ff03a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332caf6-d487-405f-a136-0dd153af6e40 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:20.439102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4827f-40a8-4360-9b3c-af96836ff03a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:20.483652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18eedde-4058-4cf4-bac8-f5aa9a8ee925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:20.483973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eedde-4058-4cf4-bac8-f5aa9a8ee925 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.484279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eedde-4058-4cf4-bac8-f5aa9a8ee925 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.484553 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eedde-4058-4cf4-bac8-f5aa9a8ee925 None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.485808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18eedde-4058-4cf4-bac8-f5aa9a8ee925 None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.487439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18eedde-4058-4cf4-bac8-f5aa9a8ee925 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.489246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 104/202] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:20 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:20.490066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18eedde-4058-4cf4-bac8-f5aa9a8ee925 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.494136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ce70d-f873-4026-beb9-5f32ae8d23a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:20.495230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ce70d-f873-4026-beb9-5f32ae8d23a4 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.495712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ce70d-f873-4026-beb9-5f32ae8d23a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.496848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ce70d-f873-4026-beb9-5f32ae8d23a4 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.499253 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ce70d-f873-4026-beb9-5f32ae8d23a4 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.500871 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:20.510449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ce70d-f873-4026-beb9-5f32ae8d23a4 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.514347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 99/203] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:20 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:20.515353 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ce70d-f873-4026-beb9-5f32ae8d23a4 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.588688 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:20.639829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ce70d-f873-4026-beb9-5f32ae8d23a4 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:20.659983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18eedde-4058-4cf4-bac8-f5aa9a8ee925 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:20.680243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71251e6c-0db1-43fa-98c3-4a497b4a2b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:20.681125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71251e6c-0db1-43fa-98c3-4a497b4a2b19 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.681125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71251e6c-0db1-43fa-98c3-4a497b4a2b19 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.681405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71251e6c-0db1-43fa-98c3-4a497b4a2b19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.682122 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71251e6c-0db1-43fa-98c3-4a497b4a2b19 None admin] RBAC: Adding query filter params (role.id=418096fb75b6406f8cb75452cb7b8edb, scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2, user.id=68bf81f77d7d40ea902bc3ef6e3a7db2) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:20.682433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71251e6c-0db1-43fa-98c3-4a497b4a2b19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.683981 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71251e6c-0db1-43fa-98c3-4a497b4a2b19 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.694535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 100/204] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:20 2022] GET /identity/v3/role_assignments?user.id=68bf81f77d7d40ea902bc3ef6e3a7db2&scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2&role.id=418096fb75b6406f8cb75452cb7b8edb => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:20.695812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71251e6c-0db1-43fa-98c3-4a497b4a2b19 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.722487 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa9cf8-9314-49c5-89ae-c23ca9a6aec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:20.723396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa9cf8-9314-49c5-89ae-c23ca9a6aec8 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.723582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa9cf8-9314-49c5-89ae-c23ca9a6aec8 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.723829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa9cf8-9314-49c5-89ae-c23ca9a6aec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.837717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71251e6c-0db1-43fa-98c3-4a497b4a2b19 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:20.872671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:20.884892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393bd9b-a805-4f95-ae50-b5327c083086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:20.885681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393bd9b-a805-4f95-ae50-b5327c083086 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.885969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393bd9b-a805-4f95-ae50-b5327c083086 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.886272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393bd9b-a805-4f95-ae50-b5327c083086 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.888407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393bd9b-a805-4f95-ae50-b5327c083086 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=7d8956a48042444687dd3ed3e39b1cd3, user.id=6410170d10c74f3ead3234ddbd115d71) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:20.888732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393bd9b-a805-4f95-ae50-b5327c083086 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.890269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393bd9b-a805-4f95-ae50-b5327c083086 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.897984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 101/205] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:20 2022] GET /identity/v3/role_assignments?user.id=6410170d10c74f3ead3234ddbd115d71&scope.project.id=7d8956a48042444687dd3ed3e39b1cd3&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:20.907637 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa9cf8-9314-49c5-89ae-c23ca9a6aec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.909462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa9cf8-9314-49c5-89ae-c23ca9a6aec8 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.923779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:21.046223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 105/206] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:21.052843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393bd9b-a805-4f95-ae50-b5327c083086 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:21.226046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393bd9b-a805-4f95-ae50-b5327c083086 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:21.275183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4acdaf-7bea-4acb-89f3-faed9e3454a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:21.275915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4acdaf-7bea-4acb-89f3-faed9e3454a1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.276193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4acdaf-7bea-4acb-89f3-faed9e3454a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.276468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4acdaf-7bea-4acb-89f3-faed9e3454a1 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.278182 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4acdaf-7bea-4acb-89f3-faed9e3454a1 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:21.279688 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4acdaf-7bea-4acb-89f3-faed9e3454a1 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:21.281690 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 102/207] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:21 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:21.287737 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa9cf8-9314-49c5-89ae-c23ca9a6aec8 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:21.468115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa9cf8-9314-49c5-89ae-c23ca9a6aec8 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:21.471738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:21.523221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543108e4-1383-4a14-9ee3-df001f51067b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:21.524760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543108e4-1383-4a14-9ee3-df001f51067b None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.525530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543108e4-1383-4a14-9ee3-df001f51067b None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.526285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543108e4-1383-4a14-9ee3-df001f51067b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.528471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543108e4-1383-4a14-9ee3-df001f51067b None admin] RBAC: Adding query filter params (role.id=83a1e8de33194004a2a672104465c819, scope.project.id=7d8956a48042444687dd3ed3e39b1cd3, user.id=6410170d10c74f3ead3234ddbd115d71) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:21.529316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543108e4-1383-4a14-9ee3-df001f51067b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:21.532681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543108e4-1383-4a14-9ee3-df001f51067b None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:21.544592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 106/208] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:21 2022] GET /identity/v3/role_assignments?user.id=6410170d10c74f3ead3234ddbd115d71&scope.project.id=7d8956a48042444687dd3ed3e39b1cd3&role.id=83a1e8de33194004a2a672104465c819 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:22.524307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cc5ac-6e61-4d44-88e1-011cf6faea26 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.525367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cc5ac-6e61-4d44-88e1-011cf6faea26 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.525868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cc5ac-6e61-4d44-88e1-011cf6faea26 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.528042 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 103/209] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:22.534040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d27c9e-8d98-4b6e-847c-139501b5175d None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.535018 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d27c9e-8d98-4b6e-847c-139501b5175d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.535547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d27c9e-8d98-4b6e-847c-139501b5175d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.556608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-47d27c9e-8d98-4b6e-847c-139501b5175d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:22.558154 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-47d27c9e-8d98-4b6e-847c-139501b5175d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:22.639888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 107/210] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:22.918298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cc5ac-6e61-4d44-88e1-011cf6faea26 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.986969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.040369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cc5ac-6e61-4d44-88e1-011cf6faea26 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.080423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe95ca-08b9-403b-89ac-b1d1e518a8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.081075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe95ca-08b9-403b-89ac-b1d1e518a8fc None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.081422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe95ca-08b9-403b-89ac-b1d1e518a8fc None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.081691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe95ca-08b9-403b-89ac-b1d1e518a8fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.093735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe95ca-08b9-403b-89ac-b1d1e518a8fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-bebe95ca-08b9-403b-89ac-b1d1e518a8fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:23.099167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:23.100087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:23.100875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 104/211] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:23.107425 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d27c9e-8d98-4b6e-847c-139501b5175d None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.217359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.224958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d27c9e-8d98-4b6e-847c-139501b5175d None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.264102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81d11a-ff19-4b0c-a96c-f042a19527ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.265120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81d11a-ff19-4b0c-a96c-f042a19527ae None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.265868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81d11a-ff19-4b0c-a96c-f042a19527ae None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.266344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81d11a-ff19-4b0c-a96c-f042a19527ae None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.267286 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81d11a-ff19-4b0c-a96c-f042a19527ae None admin] RBAC: Adding query filter params (name=member) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:23.267798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81d11a-ff19-4b0c-a96c-f042a19527ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.269413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81d11a-ff19-4b0c-a96c-f042a19527ae None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.280287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 108/212] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:23.286536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe95ca-08b9-403b-89ac-b1d1e518a8fc None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.399736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe95ca-08b9-403b-89ac-b1d1e518a8fc None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.419220 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.439791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010ef02-abab-4ca6-af94-764794558e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.440433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010ef02-abab-4ca6-af94-764794558e0a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.440706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010ef02-abab-4ca6-af94-764794558e0a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.440974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010ef02-abab-4ca6-af94-764794558e0a None admin] PATH_INFO: `/v3/users/172571166b064d75b385e775a4cc9b93` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.456643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010ef02-abab-4ca6-af94-764794558e0a None admin] RBAC: Authorizing `identity:get_user(user_id=172571166b064d75b385e775a4cc9b93)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.458195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010ef02-abab-4ca6-af94-764794558e0a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.461896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3012b7-98e8-4373-86bd-314df977a52f None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.462549 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3012b7-98e8-4373-86bd-314df977a52f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.462968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3012b7-98e8-4373-86bd-314df977a52f None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.465323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 105/213] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/users/172571166b064d75b385e775a4cc9b93 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:23.465780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 109/214] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:23.471077 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010ef02-abab-4ca6-af94-764794558e0a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.472981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca683571-f987-4abd-b717-aa3a37a9acaa None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.473479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca683571-f987-4abd-b717-aa3a37a9acaa None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.473976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca683571-f987-4abd-b717-aa3a37a9acaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.506547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ca683571-f987-4abd-b717-aa3a37a9acaa None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:23.508613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ca683571-f987-4abd-b717-aa3a37a9acaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:23.589510 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010ef02-abab-4ca6-af94-764794558e0a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.609471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 110/215] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:23.611409 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754ba65-af68-484a-96e7-2e5621ea3136 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.611873 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754ba65-af68-484a-96e7-2e5621ea3136 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.612162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754ba65-af68-484a-96e7-2e5621ea3136 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.615289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 111/216] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:23.615756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecb15d-d7c3-4909-8dd2-93f872167638 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.616195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecb15d-d7c3-4909-8dd2-93f872167638 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.616592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecb15d-d7c3-4909-8dd2-93f872167638 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.618023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 112/217] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:23.624675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffcf2a-8f1e-450b-afca-f752cbce3c2b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.625130 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffcf2a-8f1e-450b-afca-f752cbce3c2b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.625416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffcf2a-8f1e-450b-afca-f752cbce3c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.637659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156a3a3-c235-427a-805f-d980c080f102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.638288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156a3a3-c235-427a-805f-d980c080f102 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.638563 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156a3a3-c235-427a-805f-d980c080f102 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.638841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156a3a3-c235-427a-805f-d980c080f102 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.651245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-65ffcf2a-8f1e-450b-afca-f752cbce3c2b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:23.653625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-65ffcf2a-8f1e-450b-afca-f752cbce3c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:23.709260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.739792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156a3a3-c235-427a-805f-d980c080f102 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.741170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156a3a3-c235-427a-805f-d980c080f102 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.747750 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 113/218] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:23.760566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e7399-d020-4726-bc7b-8f2d6f86005f None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.760871 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e7399-d020-4726-bc7b-8f2d6f86005f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.761125 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e7399-d020-4726-bc7b-8f2d6f86005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.793239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-497e7399-d020-4726-bc7b-8f2d6f86005f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:23.794552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-497e7399-d020-4726-bc7b-8f2d6f86005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:23.815802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.868833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 106/219] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:23.875359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156a3a3-c235-427a-805f-d980c080f102 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.899220 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 114/220] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:23.899991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e7399-d020-4726-bc7b-8f2d6f86005f None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.964011 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.993770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156a3a3-c235-427a-805f-d980c080f102 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.035738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71fda3-2b62-4e15-9e36-47bc7a2d1c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.036388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71fda3-2b62-4e15-9e36-47bc7a2d1c37 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.036675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71fda3-2b62-4e15-9e36-47bc7a2d1c37 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.036987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71fda3-2b62-4e15-9e36-47bc7a2d1c37 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.038907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71fda3-2b62-4e15-9e36-47bc7a2d1c37 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.040716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71fda3-2b62-4e15-9e36-47bc7a2d1c37 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.043991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 107/221] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:24.045159 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e7399-d020-4726-bc7b-8f2d6f86005f None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.055159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71fda3-2b62-4e15-9e36-47bc7a2d1c37 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.094191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5e776-5b68-4e57-b579-20d4de7d6c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.094857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5e776-5b68-4e57-b579-20d4de7d6c00 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.095162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5e776-5b68-4e57-b579-20d4de7d6c00 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.095435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5e776-5b68-4e57-b579-20d4de7d6c00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.096619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5e776-5b68-4e57-b579-20d4de7d6c00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.098094 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5e776-5b68-4e57-b579-20d4de7d6c00 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.099772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 115/222] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:24.100554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5e776-5b68-4e57-b579-20d4de7d6c00 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.121469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:24.201182 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71fda3-2b62-4e15-9e36-47bc7a2d1c37 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.240076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5e776-5b68-4e57-b579-20d4de7d6c00 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.240861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70248c67-6d4a-498e-98f7-b9603a9bb0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.241468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70248c67-6d4a-498e-98f7-b9603a9bb0e3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.241745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70248c67-6d4a-498e-98f7-b9603a9bb0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.242022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70248c67-6d4a-498e-98f7-b9603a9bb0e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.242765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70248c67-6d4a-498e-98f7-b9603a9bb0e3 None admin] RBAC: Adding query filter params (role.id=7ccf7639a4864595b7bc589a9dc44df7, scope.project.id=7d8956a48042444687dd3ed3e39b1cd3, user.id=172571166b064d75b385e775a4cc9b93) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:24.243067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70248c67-6d4a-498e-98f7-b9603a9bb0e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.244622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70248c67-6d4a-498e-98f7-b9603a9bb0e3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.250990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 108/223] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/role_assignments?user.id=172571166b064d75b385e775a4cc9b93&scope.project.id=7d8956a48042444687dd3ed3e39b1cd3&role.id=7ccf7639a4864595b7bc589a9dc44df7 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:24.251850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70248c67-6d4a-498e-98f7-b9603a9bb0e3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.262788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:24.304500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb14ac-909e-4f6d-a05b-3bc56b7a19b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.305612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb14ac-909e-4f6d-a05b-3bc56b7a19b4 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.306020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb14ac-909e-4f6d-a05b-3bc56b7a19b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.306459 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb14ac-909e-4f6d-a05b-3bc56b7a19b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.326089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb14ac-909e-4f6d-a05b-3bc56b7a19b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-7edb14ac-909e-4f6d-a05b-3bc56b7a19b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:24.331488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:24.332793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 116/224] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:24.334349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb14ac-909e-4f6d-a05b-3bc56b7a19b4 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.404065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70248c67-6d4a-498e-98f7-b9603a9bb0e3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.446573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:24.456252 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61023a68-151f-4c6f-8e2f-fbddabfc83e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.457207 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61023a68-151f-4c6f-8e2f-fbddabfc83e4 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.457633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61023a68-151f-4c6f-8e2f-fbddabfc83e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.458131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61023a68-151f-4c6f-8e2f-fbddabfc83e4 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.459014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61023a68-151f-4c6f-8e2f-fbddabfc83e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.460685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61023a68-151f-4c6f-8e2f-fbddabfc83e4 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.485385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb14ac-909e-4f6d-a05b-3bc56b7a19b4 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.495307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 109/225] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:24 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:24.495307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61023a68-151f-4c6f-8e2f-fbddabfc83e4 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.530287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b2a4b-4710-43f4-b7d0-1b4d96b934fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.531005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b2a4b-4710-43f4-b7d0-1b4d96b934fa None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.531401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b2a4b-4710-43f4-b7d0-1b4d96b934fa None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.531685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b2a4b-4710-43f4-b7d0-1b4d96b934fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.548061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b2a4b-4710-43f4-b7d0-1b4d96b934fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-141b2a4b-4710-43f4-b7d0-1b4d96b934fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:24.553672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:24.554777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:24.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 117/226] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:24.562269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b2a4b-4710-43f4-b7d0-1b4d96b934fa None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.614635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:24.710504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61023a68-151f-4c6f-8e2f-fbddabfc83e4 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.740594 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b2a4b-4710-43f4-b7d0-1b4d96b934fa None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.765381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c043f93-0b85-4848-b80b-04a5d5c90db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.766298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c043f93-0b85-4848-b80b-04a5d5c90db3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.766652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c043f93-0b85-4848-b80b-04a5d5c90db3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.766994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c043f93-0b85-4848-b80b-04a5d5c90db3 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.767851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c043f93-0b85-4848-b80b-04a5d5c90db3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:24.768233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c043f93-0b85-4848-b80b-04a5d5c90db3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.769790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c043f93-0b85-4848-b80b-04a5d5c90db3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.781803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 110/227] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:24.787523 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c043f93-0b85-4848-b80b-04a5d5c90db3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.810885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572281f-57e6-4d3e-bd51-2f9c93462a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.811584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572281f-57e6-4d3e-bd51-2f9c93462a25 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.811867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572281f-57e6-4d3e-bd51-2f9c93462a25 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.812143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572281f-57e6-4d3e-bd51-2f9c93462a25 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.812853 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572281f-57e6-4d3e-bd51-2f9c93462a25 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:24.813271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572281f-57e6-4d3e-bd51-2f9c93462a25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.814969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572281f-57e6-4d3e-bd51-2f9c93462a25 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.823303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:24.829968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 118/228] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:24.838672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572281f-57e6-4d3e-bd51-2f9c93462a25 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.939998 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c043f93-0b85-4848-b80b-04a5d5c90db3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.975800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:24.989037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976d903-7729-4629-bf51-34c7c026198b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.989913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976d903-7729-4629-bf51-34c7c026198b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.990295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976d903-7729-4629-bf51-34c7c026198b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.990671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976d903-7729-4629-bf51-34c7c026198b None admin] PATH_INFO: `/v3/users/6410170d10c74f3ead3234ddbd115d71` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.992046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976d903-7729-4629-bf51-34c7c026198b None admin] RBAC: Authorizing `identity:get_user(user_id=6410170d10c74f3ead3234ddbd115d71)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.993634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976d903-7729-4629-bf51-34c7c026198b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.997016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 111/229] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/users/6410170d10c74f3ead3234ddbd115d71 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:25.006672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976d903-7729-4629-bf51-34c7c026198b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.018261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572281f-57e6-4d3e-bd51-2f9c93462a25 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.063717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad0fbf-99bf-4213-a65f-2e5e2423428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.064579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad0fbf-99bf-4213-a65f-2e5e2423428c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.064934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad0fbf-99bf-4213-a65f-2e5e2423428c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.065279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad0fbf-99bf-4213-a65f-2e5e2423428c None admin] PATH_INFO: `/v3/users/6410170d10c74f3ead3234ddbd115d71` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.066591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad0fbf-99bf-4213-a65f-2e5e2423428c None admin] RBAC: Authorizing `identity:get_user(user_id=6410170d10c74f3ead3234ddbd115d71)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.068320 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad0fbf-99bf-4213-a65f-2e5e2423428c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.070539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 119/230] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/users/6410170d10c74f3ead3234ddbd115d71 => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:25.079813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad0fbf-99bf-4213-a65f-2e5e2423428c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.133067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976d903-7729-4629-bf51-34c7c026198b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.172982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814b0ae-5f08-49b3-b28b-ac6b2b02ef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.174164 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814b0ae-5f08-49b3-b28b-ac6b2b02ef61 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.174562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814b0ae-5f08-49b3-b28b-ac6b2b02ef61 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.175091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814b0ae-5f08-49b3-b28b-ac6b2b02ef61 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.177112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814b0ae-5f08-49b3-b28b-ac6b2b02ef61 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.180447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814b0ae-5f08-49b3-b28b-ac6b2b02ef61 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.183541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:25.184224 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 112/231] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:25.192162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814b0ae-5f08-49b3-b28b-ac6b2b02ef61 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.271269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad0fbf-99bf-4213-a65f-2e5e2423428c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.344968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:25.361186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0fb46-f15e-46a7-9961-4aabba4b037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.362089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0fb46-f15e-46a7-9961-4aabba4b037e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.362514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0fb46-f15e-46a7-9961-4aabba4b037e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.362928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0fb46-f15e-46a7-9961-4aabba4b037e None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.365376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd0fb46-f15e-46a7-9961-4aabba4b037e None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.367035 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd0fb46-f15e-46a7-9961-4aabba4b037e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.371669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 120/232] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:25.380927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0fb46-f15e-46a7-9961-4aabba4b037e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.445039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814b0ae-5f08-49b3-b28b-ac6b2b02ef61 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.517642 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f59fca-81d7-489d-92ef-cbf251905a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.519102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f59fca-81d7-489d-92ef-cbf251905a7d None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.519732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f59fca-81d7-489d-92ef-cbf251905a7d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.520284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f59fca-81d7-489d-92ef-cbf251905a7d None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3/users/6410170d10c74f3ead3234ddbd115d71/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.524580 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f59fca-81d7-489d-92ef-cbf251905a7d None admin] RBAC: Authorizing `identity:create_grant(project_id=7d8956a48042444687dd3ed3e39b1cd3, user_id=6410170d10c74f3ead3234ddbd115d71, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.528388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f59fca-81d7-489d-92ef-cbf251905a7d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.562043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 113/233] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:25 2022] PUT /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3/users/6410170d10c74f3ead3234ddbd115d71/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:25.621103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0fb46-f15e-46a7-9961-4aabba4b037e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.680051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcfc6d-37a1-497b-9a29-eb0a8b5c2de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.681286 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcfc6d-37a1-497b-9a29-eb0a8b5c2de1 None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.681786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcfc6d-37a1-497b-9a29-eb0a8b5c2de1 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.682281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcfc6d-37a1-497b-9a29-eb0a8b5c2de1 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3/users/6410170d10c74f3ead3234ddbd115d71/roles/83a1e8de33194004a2a672104465c819` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.686212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfcfc6d-37a1-497b-9a29-eb0a8b5c2de1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d8956a48042444687dd3ed3e39b1cd3, user_id=6410170d10c74f3ead3234ddbd115d71, role_id=83a1e8de33194004a2a672104465c819)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.689139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfcfc6d-37a1-497b-9a29-eb0a8b5c2de1 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.693601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:25.716387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 121/234] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:25 2022] PUT /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3/users/6410170d10c74f3ead3234ddbd115d71/roles/83a1e8de33194004a2a672104465c819 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:34:26.573595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd070fe-d3a8-4cf7-a6c2-052b95b54421 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.575337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd070fe-d3a8-4cf7-a6c2-052b95b54421 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.575780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd070fe-d3a8-4cf7-a6c2-052b95b54421 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.578350 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 114/235] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:26.585334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb45ced-349d-4a8d-82a0-8b30d44fcea5 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.585829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb45ced-349d-4a8d-82a0-8b30d44fcea5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.586311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb45ced-349d-4a8d-82a0-8b30d44fcea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.628941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-bfb45ced-349d-4a8d-82a0-8b30d44fcea5 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:26.631076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-bfb45ced-349d-4a8d-82a0-8b30d44fcea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:26.775517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 122/236] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:27.336336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd070fe-d3a8-4cf7-a6c2-052b95b54421 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.462187 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2b24a-29b2-4ac9-9711-e9e0ed3ee38f None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.462867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2b24a-29b2-4ac9-9711-e9e0ed3ee38f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.463409 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2b24a-29b2-4ac9-9711-e9e0ed3ee38f None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.467040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 123/237] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:27.471753 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca16b8-dc15-4ec4-bbb4-468db81d69a2 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.472306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca16b8-dc15-4ec4-bbb4-468db81d69a2 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.472786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca16b8-dc15-4ec4-bbb4-468db81d69a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.485718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd070fe-d3a8-4cf7-a6c2-052b95b54421 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.513664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b9ca16b8-dc15-4ec4-bbb4-468db81d69a2 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:27.518588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b9ca16b8-dc15-4ec4-bbb4-468db81d69a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:27.518750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.555239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e48862-bf91-46a0-adf1-1f9f9da71d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.556280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e48862-bf91-46a0-adf1-1f9f9da71d04 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.556765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e48862-bf91-46a0-adf1-1f9f9da71d04 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.557239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e48862-bf91-46a0-adf1-1f9f9da71d04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e48862-bf91-46a0-adf1-1f9f9da71d04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-13e48862-bf91-46a0-adf1-1f9f9da71d04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:27.578890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:27.580313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:27.581106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:27.581106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:27.581106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:27.581106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:27.581106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:27.581106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:27.581106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:27.581106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:27.581106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:27.581106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:27.581106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:27.581106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:27.583744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 115/238] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:27.585323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfb85e-ae54-403f-8d85-2514275ea457 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.585809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfb85e-ae54-403f-8d85-2514275ea457 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.586275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfb85e-ae54-403f-8d85-2514275ea457 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.588402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 116/239] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:27.599250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfb85e-ae54-403f-8d85-2514275ea457 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.627086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 124/240] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:27.629054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db015db-607e-42b2-bc96-ef234082f153 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.629417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db015db-607e-42b2-bc96-ef234082f153 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.629993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db015db-607e-42b2-bc96-ef234082f153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.664640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-6db015db-607e-42b2-bc96-ef234082f153 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:27.665847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6db015db-607e-42b2-bc96-ef234082f153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:27.779300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.833391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfb85e-ae54-403f-8d85-2514275ea457 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.838377 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 125/241] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:27.880912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46839dd-642d-40e6-b30c-cd789a93f446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.881522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46839dd-642d-40e6-b30c-cd789a93f446 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.882004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46839dd-642d-40e6-b30c-cd789a93f446 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.882461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46839dd-642d-40e6-b30c-cd789a93f446 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.883681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46839dd-642d-40e6-b30c-cd789a93f446 None admin] RBAC: Adding query filter params (name=member) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:27.884140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46839dd-642d-40e6-b30c-cd789a93f446 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.886589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46839dd-642d-40e6-b30c-cd789a93f446 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.904888 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 117/242] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:27.912896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db015db-607e-42b2-bc96-ef234082f153 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.944887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46839dd-642d-40e6-b30c-cd789a93f446 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.037099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.150975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46839dd-642d-40e6-b30c-cd789a93f446 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.209157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db015db-607e-42b2-bc96-ef234082f153 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.216795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cde5d2-4117-4441-95e9-22f30aba187f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.217618 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cde5d2-4117-4441-95e9-22f30aba187f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.218063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cde5d2-4117-4441-95e9-22f30aba187f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.220070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cde5d2-4117-4441-95e9-22f30aba187f None admin] PATH_INFO: `/v3/domains/default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.221532 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cde5d2-4117-4441-95e9-22f30aba187f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.223470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cde5d2-4117-4441-95e9-22f30aba187f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.231151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 118/243] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:28.234324 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cde5d2-4117-4441-95e9-22f30aba187f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.287515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8494091-7ec8-4157-9fdc-1898e1fbae8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.288510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8494091-7ec8-4157-9fdc-1898e1fbae8f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.288988 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8494091-7ec8-4157-9fdc-1898e1fbae8f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.289479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8494091-7ec8-4157-9fdc-1898e1fbae8f None admin] PATH_INFO: `/v3/users/172571166b064d75b385e775a4cc9b93` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.291710 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8494091-7ec8-4157-9fdc-1898e1fbae8f None admin] RBAC: Authorizing `identity:get_user(user_id=172571166b064d75b385e775a4cc9b93)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.294850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8494091-7ec8-4157-9fdc-1898e1fbae8f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.298148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 126/244] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/users/172571166b064d75b385e775a4cc9b93 => generated 297 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:28.303271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca896f5a-c5e2-48a7-9fa8-33182646f985 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.303271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca896f5a-c5e2-48a7-9fa8-33182646f985 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.303271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca896f5a-c5e2-48a7-9fa8-33182646f985 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.303271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 127/245] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:28.304077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca896f5a-c5e2-48a7-9fa8-33182646f985 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.375493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cde5d2-4117-4441-95e9-22f30aba187f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.388031 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.398555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.423609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc97962-41af-4b59-bfd9-d05ac99aeab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.424240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc97962-41af-4b59-bfd9-d05ac99aeab5 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.424529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc97962-41af-4b59-bfd9-d05ac99aeab5 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.424793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc97962-41af-4b59-bfd9-d05ac99aeab5 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.425449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc97962-41af-4b59-bfd9-d05ac99aeab5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.426937 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc97962-41af-4b59-bfd9-d05ac99aeab5 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.459590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 119/246] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:28 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:34:28.461081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e054ba-f75a-454a-81f9-fa4e84dc2cb9 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.461380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e054ba-f75a-454a-81f9-fa4e84dc2cb9 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.461647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e054ba-f75a-454a-81f9-fa4e84dc2cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.494126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-09e054ba-f75a-454a-81f9-fa4e84dc2cb9 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:28.496456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-09e054ba-f75a-454a-81f9-fa4e84dc2cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:28.498759 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca896f5a-c5e2-48a7-9fa8-33182646f985 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.578980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31d461-af29-4e07-a6c6-23833180ff7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.580554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31d461-af29-4e07-a6c6-23833180ff7c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.581486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31d461-af29-4e07-a6c6-23833180ff7c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.582503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31d461-af29-4e07-a6c6-23833180ff7c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.587554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 120/247] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:28.588560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e054ba-f75a-454a-81f9-fa4e84dc2cb9 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.608989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31d461-af29-4e07-a6c6-23833180ff7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-9b31d461-af29-4e07-a6c6-23833180ff7c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:28.616454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:28.617562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:28.618678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:28.618678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:28.618678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:28.618678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:28.618678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:28.618678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:28.618678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:28.618678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:28.618678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:28.618678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:28.618678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:28.618678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:28.623773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 128/248] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:28.625456 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9d5fb-454c-46c6-9469-065e815936dd None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.625746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9d5fb-454c-46c6-9469-065e815936dd None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.626038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9d5fb-454c-46c6-9469-065e815936dd None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.633851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 129/249] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:28.634706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9d5fb-454c-46c6-9469-065e815936dd None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.768863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.776959 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e054ba-f75a-454a-81f9-fa4e84dc2cb9 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.824434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9d5fb-454c-46c6-9469-065e815936dd None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.837450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da6e35-53ea-47a0-b364-6314523b4b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.838513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da6e35-53ea-47a0-b364-6314523b4b45 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.838929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da6e35-53ea-47a0-b364-6314523b4b45 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.840997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da6e35-53ea-47a0-b364-6314523b4b45 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.843392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da6e35-53ea-47a0-b364-6314523b4b45 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.846028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da6e35-53ea-47a0-b364-6314523b4b45 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.849486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 121/250] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:28 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:28.851799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05a092-0629-4a07-926a-150a5ac45ce1 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.852269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05a092-0629-4a07-926a-150a5ac45ce1 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.852734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05a092-0629-4a07-926a-150a5ac45ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.861594 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.889285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ba05a092-0629-4a07-926a-150a5ac45ce1 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:28.891102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ba05a092-0629-4a07-926a-150a5ac45ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:28.904467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314911f-2b10-4f81-84fb-cc920f25397e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.905100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314911f-2b10-4f81-84fb-cc920f25397e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.905435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314911f-2b10-4f81-84fb-cc920f25397e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.905716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314911f-2b10-4f81-84fb-cc920f25397e None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.906441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314911f-2b10-4f81-84fb-cc920f25397e None admin] RBAC: Adding query filter params (name=reader) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:28.906726 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314911f-2b10-4f81-84fb-cc920f25397e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.909712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314911f-2b10-4f81-84fb-cc920f25397e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.920467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 130/251] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:28.921604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314911f-2b10-4f81-84fb-cc920f25397e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.017417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 122/252] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:29.018973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05a092-0629-4a07-926a-150a5ac45ce1 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.067154 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314911f-2b10-4f81-84fb-cc920f25397e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.131102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d34fda-4981-47b2-bad2-556a726bd2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.132218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d34fda-4981-47b2-bad2-556a726bd2f5 None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.132686 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d34fda-4981-47b2-bad2-556a726bd2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.133156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d34fda-4981-47b2-bad2-556a726bd2f5 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3/users/172571166b064d75b385e775a4cc9b93/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.137269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d34fda-4981-47b2-bad2-556a726bd2f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d8956a48042444687dd3ed3e39b1cd3, user_id=172571166b064d75b385e775a4cc9b93, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.140110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d34fda-4981-47b2-bad2-556a726bd2f5 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.175513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 131/253] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:28 2022] PUT /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3/users/172571166b064d75b385e775a4cc9b93/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:29.177900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d34fda-4981-47b2-bad2-556a726bd2f5 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.319755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.351146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.356594 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05a092-0629-4a07-926a-150a5ac45ce1 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.496249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d2175-0d8c-4976-8fe7-415cac08840a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.497340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d2175-0d8c-4976-8fe7-415cac08840a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.497826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d2175-0d8c-4976-8fe7-415cac08840a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.498272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d2175-0d8c-4976-8fe7-415cac08840a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.536317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d2175-0d8c-4976-8fe7-415cac08840a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-d52d2175-0d8c-4976-8fe7-415cac08840a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:29.553807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:29.554751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:29.555574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:29.555574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:29.555574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:29.555574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:29.555574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:29.555574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:29.555574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:29.555574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:29.555574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:29.555574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:29.555574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:29.555574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:29.565149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 123/254] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 547 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:29.566970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d2175-0d8c-4976-8fe7-415cac08840a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.611596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d34fda-4981-47b2-bad2-556a726bd2f5 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.660855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c07e7-f3a9-405b-8dee-d1db7ea0382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.662325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c07e7-f3a9-405b-8dee-d1db7ea0382f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.662810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c07e7-f3a9-405b-8dee-d1db7ea0382f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.663324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c07e7-f3a9-405b-8dee-d1db7ea0382f None admin] PATH_INFO: `/v3/users/621b6b3e24d8408da5aa083475c4c46b` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.682199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c07e7-f3a9-405b-8dee-d1db7ea0382f None admin] RBAC: Authorizing `identity:get_user(user_id=621b6b3e24d8408da5aa083475c4c46b)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.684117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c07e7-f3a9-405b-8dee-d1db7ea0382f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.686460 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 132/255] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/users/621b6b3e24d8408da5aa083475c4c46b => generated 297 bytes in 510 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:29.687517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c07e7-f3a9-405b-8dee-d1db7ea0382f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.815874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d2175-0d8c-4976-8fe7-415cac08840a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.835495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.844776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c07e7-f3a9-405b-8dee-d1db7ea0382f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.922418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07994688-0954-4003-8792-8c9e2e3900e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.925033 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07994688-0954-4003-8792-8c9e2e3900e3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.925516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07994688-0954-4003-8792-8c9e2e3900e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.925975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07994688-0954-4003-8792-8c9e2e3900e3 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.927092 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07994688-0954-4003-8792-8c9e2e3900e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:29.927562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07994688-0954-4003-8792-8c9e2e3900e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.929648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07994688-0954-4003-8792-8c9e2e3900e3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.940232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c88f4f-86fe-4c6a-b29d-4ca6aa23c538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.941120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c88f4f-86fe-4c6a-b29d-4ca6aa23c538 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.941504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c88f4f-86fe-4c6a-b29d-4ca6aa23c538 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.941877 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c88f4f-86fe-4c6a-b29d-4ca6aa23c538 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.967879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 124/256] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:29.969391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07994688-0954-4003-8792-8c9e2e3900e3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.981173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c88f4f-86fe-4c6a-b29d-4ca6aa23c538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-75c88f4f-86fe-4c6a-b29d-4ca6aa23c538 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:29.991007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:29.991999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:29.992796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:29.992796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:29.992796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:29.992796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:29.992796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:29.992796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:29.992796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:29.992796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:29.992796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:29.992796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:29.992796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:29.992796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:30.007110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 133/257] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:30.008702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c88f4f-86fe-4c6a-b29d-4ca6aa23c538 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:30.036465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:30.278954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07994688-0954-4003-8792-8c9e2e3900e3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:30.301145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c88f4f-86fe-4c6a-b29d-4ca6aa23c538 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:30.380311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86c34f-2a22-4a0d-bfe2-d42568963d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:30.381307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86c34f-2a22-4a0d-bfe2-d42568963d73 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.381758 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86c34f-2a22-4a0d-bfe2-d42568963d73 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.382216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86c34f-2a22-4a0d-bfe2-d42568963d73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:30.405894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2075f1a-3090-4511-91b6-3cda85598031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:30.406937 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2075f1a-3090-4511-91b6-3cda85598031 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.407489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2075f1a-3090-4511-91b6-3cda85598031 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.407967 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2075f1a-3090-4511-91b6-3cda85598031 None admin] PATH_INFO: `/v3/users/6410170d10c74f3ead3234ddbd115d71` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:30.416037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2075f1a-3090-4511-91b6-3cda85598031 None admin] RBAC: Authorizing `identity:get_user(user_id=6410170d10c74f3ead3234ddbd115d71)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:30.418446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2075f1a-3090-4511-91b6-3cda85598031 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:30.420784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 134/258] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:30 2022] GET /identity/v3/users/6410170d10c74f3ead3234ddbd115d71 => generated 283 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:30.421958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2075f1a-3090-4511-91b6-3cda85598031 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:30.461627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:30.545876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86c34f-2a22-4a0d-bfe2-d42568963d73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:30.548066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86c34f-2a22-4a0d-bfe2-d42568963d73 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:30.570306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:30.686775 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 125/259] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:30.688141 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86c34f-2a22-4a0d-bfe2-d42568963d73 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:30.725183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2075f1a-3090-4511-91b6-3cda85598031 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:30.806882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86c34f-2a22-4a0d-bfe2-d42568963d73 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:30.814689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac90ec0-e09c-409e-855d-6ee44a5ae957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:30.815343 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac90ec0-e09c-409e-855d-6ee44a5ae957 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.815621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac90ec0-e09c-409e-855d-6ee44a5ae957 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.815889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac90ec0-e09c-409e-855d-6ee44a5ae957 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:30.816572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac90ec0-e09c-409e-855d-6ee44a5ae957 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:30.816870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac90ec0-e09c-409e-855d-6ee44a5ae957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:30.818257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac90ec0-e09c-409e-855d-6ee44a5ae957 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:30.830423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:30.841945 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 135/260] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:30.842838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac90ec0-e09c-409e-855d-6ee44a5ae957 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:30.854540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b08ee-8b1f-454c-87bf-162cdc5f3f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:30.855878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b08ee-8b1f-454c-87bf-162cdc5f3f86 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.856376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b08ee-8b1f-454c-87bf-162cdc5f3f86 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.856895 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b08ee-8b1f-454c-87bf-162cdc5f3f86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.001325 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b08ee-8b1f-454c-87bf-162cdc5f3f86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:31.004166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b08ee-8b1f-454c-87bf-162cdc5f3f86 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:31.123671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:31.183853 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac90ec0-e09c-409e-855d-6ee44a5ae957 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.237493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9ba3b-f541-4747-8133-d958441601cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.238585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9ba3b-f541-4747-8133-d958441601cd None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.239087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9ba3b-f541-4747-8133-d958441601cd None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.239646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9ba3b-f541-4747-8133-d958441601cd None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.241754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9ba3b-f541-4747-8133-d958441601cd None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:31.244994 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9ba3b-f541-4747-8133-d958441601cd None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:31.248819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 136/261] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:30 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:31.250231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9ba3b-f541-4747-8133-d958441601cd None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.354629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 126/262] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:31.356420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b08ee-8b1f-454c-87bf-162cdc5f3f86 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.423682 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:31.501903 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9ba3b-f541-4747-8133-d958441601cd None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.534760 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b08ee-8b1f-454c-87bf-162cdc5f3f86 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.581246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66855b27-7b9c-4ed5-9f02-d1bf3be68406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.581908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66855b27-7b9c-4ed5-9f02-d1bf3be68406 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.582188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66855b27-7b9c-4ed5-9f02-d1bf3be68406 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.582466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66855b27-7b9c-4ed5-9f02-d1bf3be68406 None admin] PATH_INFO: `/v3/users/6410170d10c74f3ead3234ddbd115d71` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.583810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66855b27-7b9c-4ed5-9f02-d1bf3be68406 None admin] RBAC: Authorizing `identity:get_user(user_id=6410170d10c74f3ead3234ddbd115d71)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:31.585295 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66855b27-7b9c-4ed5-9f02-d1bf3be68406 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:31.592909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c11ba-4634-4913-ac2e-1c99c4da9b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.593531 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c11ba-4634-4913-ac2e-1c99c4da9b8e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.593814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c11ba-4634-4913-ac2e-1c99c4da9b8e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.594087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c11ba-4634-4913-ac2e-1c99c4da9b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.594838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c11ba-4634-4913-ac2e-1c99c4da9b8e None admin] RBAC: Adding query filter params (role.id=418096fb75b6406f8cb75452cb7b8edb, scope.project.id=7d8956a48042444687dd3ed3e39b1cd3, user.id=621b6b3e24d8408da5aa083475c4c46b) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:31.595116 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 137/263] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/users/6410170d10c74f3ead3234ddbd115d71 => generated 283 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:31.596086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66855b27-7b9c-4ed5-9f02-d1bf3be68406 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.596617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c11ba-4634-4913-ac2e-1c99c4da9b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:31.598141 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c11ba-4634-4913-ac2e-1c99c4da9b8e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:31.604504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 127/264] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/role_assignments?user.id=621b6b3e24d8408da5aa083475c4c46b&scope.project.id=7d8956a48042444687dd3ed3e39b1cd3&role.id=418096fb75b6406f8cb75452cb7b8edb => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:31.606173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c11ba-4634-4913-ac2e-1c99c4da9b8e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.649489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:31.715707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:31.747620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c11ba-4634-4913-ac2e-1c99c4da9b8e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.820993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3a73d-6a14-46d2-9e71-d5f907c4ba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.821637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3a73d-6a14-46d2-9e71-d5f907c4ba25 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.821911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3a73d-6a14-46d2-9e71-d5f907c4ba25 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.822349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3a73d-6a14-46d2-9e71-d5f907c4ba25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.833434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66855b27-7b9c-4ed5-9f02-d1bf3be68406 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.891195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac08d3-c290-4203-81e1-cabebe2f78a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.892218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac08d3-c290-4203-81e1-cabebe2f78a7 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.892718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac08d3-c290-4203-81e1-cabebe2f78a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.893187 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac08d3-c290-4203-81e1-cabebe2f78a7 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.894647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac08d3-c290-4203-81e1-cabebe2f78a7 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:31.896987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac08d3-c290-4203-81e1-cabebe2f78a7 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:31.898970 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 138/265] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:31.901688 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4314654-f0d3-43cc-882e-347a270b79a4 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.902315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4314654-f0d3-43cc-882e-347a270b79a4 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.902862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4314654-f0d3-43cc-882e-347a270b79a4 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.904697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 139/266] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:31.909931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4314654-f0d3-43cc-882e-347a270b79a4 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.046005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3a73d-6a14-46d2-9e71-d5f907c4ba25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.049288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3a73d-6a14-46d2-9e71-d5f907c4ba25 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.127938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:32.168601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4314654-f0d3-43cc-882e-347a270b79a4 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.249291 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:32.293133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 128/267] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:32.293734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d57051-60bf-4b73-bd17-74a4aec85e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.294497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d57051-60bf-4b73-bd17-74a4aec85e3d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.294882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d57051-60bf-4b73-bd17-74a4aec85e3d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.295289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d57051-60bf-4b73-bd17-74a4aec85e3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.296151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d57051-60bf-4b73-bd17-74a4aec85e3d None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=7d8956a48042444687dd3ed3e39b1cd3, user.id=6410170d10c74f3ead3234ddbd115d71) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:32.296573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d57051-60bf-4b73-bd17-74a4aec85e3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.297174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32949d12-4abb-408a-936f-24a99cf409a9 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.297834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32949d12-4abb-408a-936f-24a99cf409a9 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.298346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d57051-60bf-4b73-bd17-74a4aec85e3d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.298719 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32949d12-4abb-408a-936f-24a99cf409a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.327787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 140/268] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/role_assignments?user.id=6410170d10c74f3ead3234ddbd115d71&scope.project.id=7d8956a48042444687dd3ed3e39b1cd3&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 608 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:32.327787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d57051-60bf-4b73-bd17-74a4aec85e3d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.347170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-32949d12-4abb-408a-936f-24a99cf409a9 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:32.348592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-32949d12-4abb-408a-936f-24a99cf409a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:32.510892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 129/269] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:32.512452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2ab2c-702d-48c4-a8fe-939a7abfe13a None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.512836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2ab2c-702d-48c4-a8fe-939a7abfe13a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.513214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2ab2c-702d-48c4-a8fe-939a7abfe13a None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.514836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 130/270] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:32.524886 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a8d62-f3b8-4b73-9f46-d09243427839 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.525342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a8d62-f3b8-4b73-9f46-d09243427839 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.525723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a8d62-f3b8-4b73-9f46-d09243427839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.550863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d57051-60bf-4b73-bd17-74a4aec85e3d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.561775 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-192a8d62-f3b8-4b73-9f46-d09243427839 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:32.563373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-192a8d62-f3b8-4b73-9f46-d09243427839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:32.635168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953d31f-695c-4894-8b49-063ccf1f9708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.636114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953d31f-695c-4894-8b49-063ccf1f9708 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.636436 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953d31f-695c-4894-8b49-063ccf1f9708 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.636709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953d31f-695c-4894-8b49-063ccf1f9708 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.642583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953d31f-695c-4894-8b49-063ccf1f9708 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.644138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953d31f-695c-4894-8b49-063ccf1f9708 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.648068 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 141/271] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:32.661691 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953d31f-695c-4894-8b49-063ccf1f9708 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.700607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:32.734241 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 131/272] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:32.821648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2ad49-4be3-44e2-9d1b-aa93fc00728b None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.822059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2ad49-4be3-44e2-9d1b-aa93fc00728b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.822490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2ad49-4be3-44e2-9d1b-aa93fc00728b None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.826668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 132/273] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:32.835866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904fca0-aaa4-44bf-a04a-959a53e392a1 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.836327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904fca0-aaa4-44bf-a04a-959a53e392a1 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.836793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904fca0-aaa4-44bf-a04a-959a53e392a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.848558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953d31f-695c-4894-8b49-063ccf1f9708 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.891407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-c904fca0-aaa4-44bf-a04a-959a53e392a1 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:32.893996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c904fca0-aaa4-44bf-a04a-959a53e392a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:32.941200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fcbac-783b-49f2-8ad2-b30671c8651e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.942185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fcbac-783b-49f2-8ad2-b30671c8651e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.942752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fcbac-783b-49f2-8ad2-b30671c8651e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.943366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fcbac-783b-49f2-8ad2-b30671c8651e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.944940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fcbac-783b-49f2-8ad2-b30671c8651e None admin] RBAC: Adding query filter params (role.id=83a1e8de33194004a2a672104465c819, scope.project.id=7d8956a48042444687dd3ed3e39b1cd3, user.id=6410170d10c74f3ead3234ddbd115d71) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:32.945467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fcbac-783b-49f2-8ad2-b30671c8651e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.948435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fcbac-783b-49f2-8ad2-b30671c8651e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.961373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 142/274] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/role_assignments?user.id=6410170d10c74f3ead3234ddbd115d71&scope.project.id=7d8956a48042444687dd3ed3e39b1cd3&role.id=83a1e8de33194004a2a672104465c819 => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:32.965233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fcbac-783b-49f2-8ad2-b30671c8651e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.987927 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:33.097003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 133/275] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:33.188077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:33.220760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fcbac-783b-49f2-8ad2-b30671c8651e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:33.272682 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904fca0-aaa4-44bf-a04a-959a53e392a1 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.278894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8777ac3-b9a1-4da4-a4b8-5387c1fa9d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:33.279929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8777ac3-b9a1-4da4-a4b8-5387c1fa9d55 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.280386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8777ac3-b9a1-4da4-a4b8-5387c1fa9d55 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.280830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8777ac3-b9a1-4da4-a4b8-5387c1fa9d55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.293797 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8777ac3-b9a1-4da4-a4b8-5387c1fa9d55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-e8777ac3-b9a1-4da4-a4b8-5387c1fa9d55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:33.299951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:33.300857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:33.301635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:33.301635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:33.301635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:33.301635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:33.301635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:33.301635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:33.301635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:33.301635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:33.301635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:33.301635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:33.301635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:33.301635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:33.304236 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 143/276] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:33.310784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8777ac3-b9a1-4da4-a4b8-5387c1fa9d55 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.498672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904fca0-aaa4-44bf-a04a-959a53e392a1 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:33.539150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8777ac3-b9a1-4da4-a4b8-5387c1fa9d55 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:33.567182 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b8fa6-bea4-4938-af84-35269e858a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:33.568064 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b8fa6-bea4-4938-af84-35269e858a91 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.568343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b8fa6-bea4-4938-af84-35269e858a91 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.568612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b8fa6-bea4-4938-af84-35269e858a91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.569792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b8fa6-bea4-4938-af84-35269e858a91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:33.571371 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b8fa6-bea4-4938-af84-35269e858a91 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:33.573286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 134/277] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:33.574383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b8fa6-bea4-4938-af84-35269e858a91 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.590057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768403f-a2a0-4bac-9e02-f10006032072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:33.591108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768403f-a2a0-4bac-9e02-f10006032072 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.591740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768403f-a2a0-4bac-9e02-f10006032072 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.592233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768403f-a2a0-4bac-9e02-f10006032072 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.593470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768403f-a2a0-4bac-9e02-f10006032072 None admin] RBAC: Adding query filter params (name=member) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:33.594031 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768403f-a2a0-4bac-9e02-f10006032072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:33.595700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768403f-a2a0-4bac-9e02-f10006032072 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:33.614270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 144/278] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:33.614270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768403f-a2a0-4bac-9e02-f10006032072 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.619676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:33.651479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:33.726878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b8fa6-bea4-4938-af84-35269e858a91 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:33.769615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fd2f5-7d1d-4074-a959-c2e170e7f29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:33.770171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fd2f5-7d1d-4074-a959-c2e170e7f29f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.770447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fd2f5-7d1d-4074-a959-c2e170e7f29f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.770723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fd2f5-7d1d-4074-a959-c2e170e7f29f None admin] PATH_INFO: `/v3/roles/member` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.784325 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326fd2f5-7d1d-4074-a959-c2e170e7f29f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-326fd2f5-7d1d-4074-a959-c2e170e7f29f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:33.789289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:33.791034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:33.793685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 135/279] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:33.798051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fd2f5-7d1d-4074-a959-c2e170e7f29f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.809246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768403f-a2a0-4bac-9e02-f10006032072 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:33.873643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7652f0-5a81-4cde-93b5-d30bf92397ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:33.874688 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7652f0-5a81-4cde-93b5-d30bf92397ab None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.875168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7652f0-5a81-4cde-93b5-d30bf92397ab None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.875644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7652f0-5a81-4cde-93b5-d30bf92397ab None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.876883 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7652f0-5a81-4cde-93b5-d30bf92397ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:33.879595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7652f0-5a81-4cde-93b5-d30bf92397ab None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:33.919180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 145/280] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:33 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:33.919180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7652f0-5a81-4cde-93b5-d30bf92397ab None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.955974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:33.960958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fd2f5-7d1d-4074-a959-c2e170e7f29f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.007445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1777305-8f05-4d95-989c-4473b4f4cde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:34.008803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1777305-8f05-4d95-989c-4473b4f4cde2 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.009204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1777305-8f05-4d95-989c-4473b4f4cde2 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.009599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1777305-8f05-4d95-989c-4473b4f4cde2 None admin] PATH_INFO: `/v3/users/d55db7b2dbe240a7914b4f2ed34f74de` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.034727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1777305-8f05-4d95-989c-4473b4f4cde2 None admin] RBAC: Authorizing `identity:get_user(user_id=d55db7b2dbe240a7914b4f2ed34f74de)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:34.037555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1777305-8f05-4d95-989c-4473b4f4cde2 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:34.040925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 136/281] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:33 2022] GET /identity/v3/users/d55db7b2dbe240a7914b4f2ed34f74de => generated 293 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:34.043370 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:34.050539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1777305-8f05-4d95-989c-4473b4f4cde2 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:34.149042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7652f0-5a81-4cde-93b5-d30bf92397ab None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.221514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39975764-85f9-4660-93d6-333be3069ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:34.222552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39975764-85f9-4660-93d6-333be3069ed7 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.222975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39975764-85f9-4660-93d6-333be3069ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.223425 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39975764-85f9-4660-93d6-333be3069ed7 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.224277 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39975764-85f9-4660-93d6-333be3069ed7 None admin] RBAC: Adding query filter params (name=member) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:34.224581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39975764-85f9-4660-93d6-333be3069ed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:34.226007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39975764-85f9-4660-93d6-333be3069ed7 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:34.237661 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 146/282] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:34.238971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb52512-4db8-4f91-9c8e-c0b748f6004f None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.239393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb52512-4db8-4f91-9c8e-c0b748f6004f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.239630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb52512-4db8-4f91-9c8e-c0b748f6004f None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.240982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 147/283] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:34.247417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e68535-dd84-4a3a-a9db-3900ee1e0fa3 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.247717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e68535-dd84-4a3a-a9db-3900ee1e0fa3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.247975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e68535-dd84-4a3a-a9db-3900ee1e0fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.256214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1777305-8f05-4d95-989c-4473b4f4cde2 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.272981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-48e68535-dd84-4a3a-a9db-3900ee1e0fa3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:34.274193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-48e68535-dd84-4a3a-a9db-3900ee1e0fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:34.327177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d001991-be89-45db-a186-e0bef0967477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:34.328131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d001991-be89-45db-a186-e0bef0967477 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.328586 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d001991-be89-45db-a186-e0bef0967477 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.329056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d001991-be89-45db-a186-e0bef0967477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.330451 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d001991-be89-45db-a186-e0bef0967477 None admin] RBAC: Adding query filter params (role.id=7ccf7639a4864595b7bc589a9dc44df7, user.id=d55db7b2dbe240a7914b4f2ed34f74de, scope.system=all) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:34.330961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d001991-be89-45db-a186-e0bef0967477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:34.333722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d001991-be89-45db-a186-e0bef0967477 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:34.345079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 137/284] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Aug 31 09:34:34 2022] GET /identity/v3/role_assignments?user.id=d55db7b2dbe240a7914b4f2ed34f74de&scope.system=all&role.id=7ccf7639a4864595b7bc589a9dc44df7 => generated 226 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:34.346563 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d001991-be89-45db-a186-e0bef0967477 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:34.380729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 148/285] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:34.398230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:34.559750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d001991-be89-45db-a186-e0bef0967477 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.635947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da039ba6-fb1b-4d28-9c8d-71147511f828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:34.637185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da039ba6-fb1b-4d28-9c8d-71147511f828 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.637773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da039ba6-fb1b-4d28-9c8d-71147511f828 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.638222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da039ba6-fb1b-4d28-9c8d-71147511f828 None admin] PATH_INFO: `/v3/users/172571166b064d75b385e775a4cc9b93` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.641492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da039ba6-fb1b-4d28-9c8d-71147511f828 None admin] RBAC: Authorizing `identity:get_user(user_id=172571166b064d75b385e775a4cc9b93)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:34.644385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da039ba6-fb1b-4d28-9c8d-71147511f828 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:34.647928 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 138/286] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:34 2022] GET /identity/v3/users/172571166b064d75b385e775a4cc9b93 => generated 297 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:34.657507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e68535-dd84-4a3a-a9db-3900ee1e0fa3 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:34.704517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da039ba6-fb1b-4d28-9c8d-71147511f828 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:34.774348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:34.884468 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e68535-dd84-4a3a-a9db-3900ee1e0fa3 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.904827 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:34.917502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da039ba6-fb1b-4d28-9c8d-71147511f828 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.977840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3902080-487b-4abb-9362-662db5475d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:34.978473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3902080-487b-4abb-9362-662db5475d9c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.978751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3902080-487b-4abb-9362-662db5475d9c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.979018 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3902080-487b-4abb-9362-662db5475d9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.005685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee354019-a88d-497f-bb26-7b363f6d364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.006861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354019-a88d-497f-bb26-7b363f6d364f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.007395 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354019-a88d-497f-bb26-7b363f6d364f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.007860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354019-a88d-497f-bb26-7b363f6d364f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.049521 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee354019-a88d-497f-bb26-7b363f6d364f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-ee354019-a88d-497f-bb26-7b363f6d364f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:35.068610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:35.070341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:35.073497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 139/287] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:35.085091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee354019-a88d-497f-bb26-7b363f6d364f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.150178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3902080-487b-4abb-9362-662db5475d9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.151701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3902080-487b-4abb-9362-662db5475d9c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.269292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.331293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee354019-a88d-497f-bb26-7b363f6d364f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.387039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 149/288] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 730 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:35.391617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81f694-03c3-4f05-8e1f-0265c3b2c55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.392572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81f694-03c3-4f05-8e1f-0265c3b2c55e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.392891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81f694-03c3-4f05-8e1f-0265c3b2c55e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.393165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81f694-03c3-4f05-8e1f-0265c3b2c55e None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.393863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81f694-03c3-4f05-8e1f-0265c3b2c55e None admin] RBAC: Adding query filter params (name=reader) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:35.394166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81f694-03c3-4f05-8e1f-0265c3b2c55e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.395631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81f694-03c3-4f05-8e1f-0265c3b2c55e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.406152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 140/289] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:35.412251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3902080-487b-4abb-9362-662db5475d9c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.421839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81f694-03c3-4f05-8e1f-0265c3b2c55e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.451884 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.582697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3902080-487b-4abb-9362-662db5475d9c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.658916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81f694-03c3-4f05-8e1f-0265c3b2c55e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.660892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589ea91-7633-436b-9364-6e0b0e729704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.661569 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589ea91-7633-436b-9364-6e0b0e729704 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.661841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589ea91-7633-436b-9364-6e0b0e729704 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.662111 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589ea91-7633-436b-9364-6e0b0e729704 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.663476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589ea91-7633-436b-9364-6e0b0e729704 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.664927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589ea91-7633-436b-9364-6e0b0e729704 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.666880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 150/290] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:35.673935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589ea91-7633-436b-9364-6e0b0e729704 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.719509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.750348 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c9f37-32c9-45c2-9efa-cd12ca954ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.751021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c9f37-32c9-45c2-9efa-cd12ca954ed6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.751327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c9f37-32c9-45c2-9efa-cd12ca954ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.751599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c9f37-32c9-45c2-9efa-cd12ca954ed6 None admin] PATH_INFO: `/v3/users/621b6b3e24d8408da5aa083475c4c46b` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.754270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c9f37-32c9-45c2-9efa-cd12ca954ed6 None admin] RBAC: Authorizing `identity:get_user(user_id=621b6b3e24d8408da5aa083475c4c46b)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.755864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c9f37-32c9-45c2-9efa-cd12ca954ed6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.758684 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 141/291] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/users/621b6b3e24d8408da5aa083475c4c46b => generated 297 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:35.768196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c9f37-32c9-45c2-9efa-cd12ca954ed6 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.886320 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589ea91-7633-436b-9364-6e0b0e729704 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.935534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.958917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c682449-c0c8-40c0-8b09-93608f2c5e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.959561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c682449-c0c8-40c0-8b09-93608f2c5e60 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.959842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c682449-c0c8-40c0-8b09-93608f2c5e60 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.960114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c682449-c0c8-40c0-8b09-93608f2c5e60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.960854 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c682449-c0c8-40c0-8b09-93608f2c5e60 None admin] RBAC: Adding query filter params (role.id=7ccf7639a4864595b7bc589a9dc44df7, scope.project.id=7d8956a48042444687dd3ed3e39b1cd3, user.id=172571166b064d75b385e775a4cc9b93) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:35.961163 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c682449-c0c8-40c0-8b09-93608f2c5e60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.962606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c682449-c0c8-40c0-8b09-93608f2c5e60 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.977318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c9f37-32c9-45c2-9efa-cd12ca954ed6 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.978322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 151/292] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/role_assignments?user.id=172571166b064d75b385e775a4cc9b93&scope.project.id=7d8956a48042444687dd3ed3e39b1cd3&role.id=7ccf7639a4864595b7bc589a9dc44df7 => generated 608 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:36.034162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db88dc-b570-4b32-9b07-18f3b4202ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.034820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db88dc-b570-4b32-9b07-18f3b4202ca3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.035099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db88dc-b570-4b32-9b07-18f3b4202ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.035393 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db88dc-b570-4b32-9b07-18f3b4202ca3 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.036631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db88dc-b570-4b32-9b07-18f3b4202ca3 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.038262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db88dc-b570-4b32-9b07-18f3b4202ca3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.040371 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 142/293] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:36.046822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c682449-c0c8-40c0-8b09-93608f2c5e60 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.178313 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.188577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c682449-c0c8-40c0-8b09-93608f2c5e60 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.226725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f3491-15c2-4585-92e9-bf68bf8625d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.227449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f3491-15c2-4585-92e9-bf68bf8625d6 None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.227732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f3491-15c2-4585-92e9-bf68bf8625d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.228007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f3491-15c2-4585-92e9-bf68bf8625d6 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3/users/621b6b3e24d8408da5aa083475c4c46b/roles/418096fb75b6406f8cb75452cb7b8edb` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.230229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f3491-15c2-4585-92e9-bf68bf8625d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d8956a48042444687dd3ed3e39b1cd3, user_id=621b6b3e24d8408da5aa083475c4c46b, role_id=418096fb75b6406f8cb75452cb7b8edb)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.231788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f3491-15c2-4585-92e9-bf68bf8625d6 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.256358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 152/294] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:36 2022] PUT /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3/users/621b6b3e24d8408da5aa083475c4c46b/roles/418096fb75b6406f8cb75452cb7b8edb => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:36.772552 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f84dc-8f28-45ae-8dd6-5f823a42653a None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.773322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f84dc-8f28-45ae-8dd6-5f823a42653a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.773322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f84dc-8f28-45ae-8dd6-5f823a42653a None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.774893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 143/295] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:36.781243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa802e-3903-40cb-a603-66f99a7747cd None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.781615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa802e-3903-40cb-a603-66f99a7747cd None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.781853 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa802e-3903-40cb-a603-66f99a7747cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.810026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-6baa802e-3903-40cb-a603-66f99a7747cd None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:36.811305 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6baa802e-3903-40cb-a603-66f99a7747cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:36.890960 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 153/296] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:36.913957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b973d-81eb-4cad-a0ec-1f92e6cae46b None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.914265 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b973d-81eb-4cad-a0ec-1f92e6cae46b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.914503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b973d-81eb-4cad-a0ec-1f92e6cae46b None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.916045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 144/297] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:36.921789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964580ce-0fab-4f2b-81cb-046a5d8bf9df None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.922075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964580ce-0fab-4f2b-81cb-046a5d8bf9df None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.922343 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964580ce-0fab-4f2b-81cb-046a5d8bf9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.948638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-964580ce-0fab-4f2b-81cb-046a5d8bf9df None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:36.951444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-964580ce-0fab-4f2b-81cb-046a5d8bf9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:37.065849 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 154/298] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:37.092562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f27fb3-5937-46e3-8f6c-1727dcc77de2 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.092922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f27fb3-5937-46e3-8f6c-1727dcc77de2 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.093148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f27fb3-5937-46e3-8f6c-1727dcc77de2 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.094465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 145/299] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:37.100152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ff5af-8a6b-44d5-9421-0ee8b213cccb None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.100437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ff5af-8a6b-44d5-9421-0ee8b213cccb None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.100783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ff5af-8a6b-44d5-9421-0ee8b213cccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.115567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.129590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-dc3ff5af-8a6b-44d5-9421-0ee8b213cccb None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:37.130813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-dc3ff5af-8a6b-44d5-9421-0ee8b213cccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:37.189501 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f27fb3-5937-46e3-8f6c-1727dcc77de2 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.264748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 155/300] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:37.304003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f27fb3-5937-46e3-8f6c-1727dcc77de2 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.348004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8192935-c700-4eaa-9ad5-52f0db9eae3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.348943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8192935-c700-4eaa-9ad5-52f0db9eae3a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.349367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8192935-c700-4eaa-9ad5-52f0db9eae3a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8192935-c700-4eaa-9ad5-52f0db9eae3a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.361994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8192935-c700-4eaa-9ad5-52f0db9eae3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-d8192935-c700-4eaa-9ad5-52f0db9eae3a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:37.367734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:37.368845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:37.369833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:37.369833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:37.369833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:37.369833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:37.369833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:37.369833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:37.369833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:37.369833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:37.369833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:37.369833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:37.369833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:37.369833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:37.372441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 146/301] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:37.387239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ff5af-8a6b-44d5-9421-0ee8b213cccb None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.398240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8192935-c700-4eaa-9ad5-52f0db9eae3a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.412842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.498929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ff5af-8a6b-44d5-9421-0ee8b213cccb None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.537154 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6828a0-7dae-4ae4-9eba-9d2bd365151c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.538286 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6828a0-7dae-4ae4-9eba-9d2bd365151c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.538763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6828a0-7dae-4ae4-9eba-9d2bd365151c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.539306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6828a0-7dae-4ae4-9eba-9d2bd365151c None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.540200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6828a0-7dae-4ae4-9eba-9d2bd365151c None admin] RBAC: Adding query filter params (name=reader) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:37.540792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6828a0-7dae-4ae4-9eba-9d2bd365151c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.542430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6828a0-7dae-4ae4-9eba-9d2bd365151c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.545513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.553070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 156/302] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:37.555423 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6828a0-7dae-4ae4-9eba-9d2bd365151c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.583311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8192935-c700-4eaa-9ad5-52f0db9eae3a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.645168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55e51a-948b-4f13-9a93-9d255b36bd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.646549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55e51a-948b-4f13-9a93-9d255b36bd26 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.647284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55e51a-948b-4f13-9a93-9d255b36bd26 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.647988 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55e51a-948b-4f13-9a93-9d255b36bd26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.667229 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55e51a-948b-4f13-9a93-9d255b36bd26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:37.674438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6828a0-7dae-4ae4-9eba-9d2bd365151c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-2d55e51a-948b-4f13-9a93-9d255b36bd26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:37.678163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:37.679846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:37.682009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 147/303] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:37.684716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55e51a-948b-4f13-9a93-9d255b36bd26 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.727357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15024d94-96e2-45fc-b48d-b967808d2177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.728761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15024d94-96e2-45fc-b48d-b967808d2177 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.729227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15024d94-96e2-45fc-b48d-b967808d2177 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.729724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15024d94-96e2-45fc-b48d-b967808d2177 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.732670 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15024d94-96e2-45fc-b48d-b967808d2177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.735545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15024d94-96e2-45fc-b48d-b967808d2177 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.738605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 157/304] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:37.739819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15024d94-96e2-45fc-b48d-b967808d2177 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.787584 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.847030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.858855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55e51a-948b-4f13-9a93-9d255b36bd26 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.912389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15024d94-96e2-45fc-b48d-b967808d2177 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.922426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1035d-bb15-4f0f-bd08-2047df8c2666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.924809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1035d-bb15-4f0f-bd08-2047df8c2666 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.924809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1035d-bb15-4f0f-bd08-2047df8c2666 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.924809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1035d-bb15-4f0f-bd08-2047df8c2666 None admin] PATH_INFO: `/v3/users/95fd141d4f704559878fe01dbf9878d5` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.949836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1035d-bb15-4f0f-bd08-2047df8c2666 None admin] RBAC: Authorizing `identity:get_user(user_id=95fd141d4f704559878fe01dbf9878d5)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.951868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1035d-bb15-4f0f-bd08-2047df8c2666 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.958865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 148/305] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/users/95fd141d4f704559878fe01dbf9878d5 => generated 293 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:37.958865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1035d-bb15-4f0f-bd08-2047df8c2666 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.967318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c770a4-4a1d-4c2b-abf1-180ec51fbf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.967964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c770a4-4a1d-4c2b-abf1-180ec51fbf38 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.968236 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c770a4-4a1d-4c2b-abf1-180ec51fbf38 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.968506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c770a4-4a1d-4c2b-abf1-180ec51fbf38 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.969188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c770a4-4a1d-4c2b-abf1-180ec51fbf38 None admin] RBAC: Adding query filter params (name=member) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:37.969511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c770a4-4a1d-4c2b-abf1-180ec51fbf38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.970906 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c770a4-4a1d-4c2b-abf1-180ec51fbf38 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.980881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 158/306] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:37.981746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c770a4-4a1d-4c2b-abf1-180ec51fbf38 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.102843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1035d-bb15-4f0f-bd08-2047df8c2666 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.128871 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c770a4-4a1d-4c2b-abf1-180ec51fbf38 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.147736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:38.170775 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546578c7-6375-4d04-aa90-0121b23c4944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.172261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546578c7-6375-4d04-aa90-0121b23c4944 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.172884 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546578c7-6375-4d04-aa90-0121b23c4944 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.173444 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546578c7-6375-4d04-aa90-0121b23c4944 None admin] PATH_INFO: `/v3/groups` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.174282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546578c7-6375-4d04-aa90-0121b23c4944 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.176315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546578c7-6375-4d04-aa90-0121b23c4944 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.178526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:38.191808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc799c-f384-4688-9450-598d7486c050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.192586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc799c-f384-4688-9450-598d7486c050 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.193129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc799c-f384-4688-9450-598d7486c050 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.193688 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc799c-f384-4688-9450-598d7486c050 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.195202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc799c-f384-4688-9450-598d7486c050 None admin] RBAC: Adding query filter params (role.id=418096fb75b6406f8cb75452cb7b8edb, user.id=95fd141d4f704559878fe01dbf9878d5, scope.system=all) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:38.195749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc799c-f384-4688-9450-598d7486c050 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.198489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc799c-f384-4688-9450-598d7486c050 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.199432 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 149/307] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Aug 31 09:34:37 2022] POST /identity/v3/groups => generated 229 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:38.200553 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546578c7-6375-4d04-aa90-0121b23c4944 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.207186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 159/308] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/role_assignments?user.id=95fd141d4f704559878fe01dbf9878d5&scope.system=all&role.id=418096fb75b6406f8cb75452cb7b8edb => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:38.389866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546578c7-6375-4d04-aa90-0121b23c4944 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.450413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd108a-14c1-4e5a-9dae-731762da627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.451073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd108a-14c1-4e5a-9dae-731762da627f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.451386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd108a-14c1-4e5a-9dae-731762da627f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.451660 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd108a-14c1-4e5a-9dae-731762da627f None admin] PATH_INFO: `/v3/users/d55db7b2dbe240a7914b4f2ed34f74de` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.452905 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd108a-14c1-4e5a-9dae-731762da627f None admin] RBAC: Authorizing `identity:get_user(user_id=d55db7b2dbe240a7914b4f2ed34f74de)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.454399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd108a-14c1-4e5a-9dae-731762da627f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.456421 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 150/309] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:38 2022] GET /identity/v3/users/d55db7b2dbe240a7914b4f2ed34f74de => generated 293 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:38.462255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc799c-f384-4688-9450-598d7486c050 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.593975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc799c-f384-4688-9450-598d7486c050 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.633124 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fa0c6-456c-4ebc-b1ea-adc487ee2091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.633864 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fa0c6-456c-4ebc-b1ea-adc487ee2091 None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.634147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fa0c6-456c-4ebc-b1ea-adc487ee2091 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.634424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fa0c6-456c-4ebc-b1ea-adc487ee2091 None admin] PATH_INFO: `/v3/system/users/d55db7b2dbe240a7914b4f2ed34f74de/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.635972 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fa0c6-456c-4ebc-b1ea-adc487ee2091 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d55db7b2dbe240a7914b4f2ed34f74de, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.638364 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fa0c6-456c-4ebc-b1ea-adc487ee2091 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.645591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf68c45-f55a-4bf7-ac8c-4e7ef889c097 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.645874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf68c45-f55a-4bf7-ac8c-4e7ef889c097 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.646143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf68c45-f55a-4bf7-ac8c-4e7ef889c097 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.647911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 151/310] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:38.650950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 160/311] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Aug 31 09:34:38 2022] PUT /identity/v3/system/users/d55db7b2dbe240a7914b4f2ed34f74de/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:38.663011 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014b211-0d0d-443c-b97d-dba701830c9b None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.663362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014b211-0d0d-443c-b97d-dba701830c9b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.663645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014b211-0d0d-443c-b97d-dba701830c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.692777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a014b211-0d0d-443c-b97d-dba701830c9b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:38.694486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a014b211-0d0d-443c-b97d-dba701830c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:38.758786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:38.785954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 152/312] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:39.070027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fa0c6-456c-4ebc-b1ea-adc487ee2091 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.137455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.188604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fa0c6-456c-4ebc-b1ea-adc487ee2091 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.226054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fd4d9-41bf-4b6e-9b00-89ee63ce46c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.226697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fd4d9-41bf-4b6e-9b00-89ee63ce46c6 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.226980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fd4d9-41bf-4b6e-9b00-89ee63ce46c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.227286 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fd4d9-41bf-4b6e-9b00-89ee63ce46c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.238724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fd4d9-41bf-4b6e-9b00-89ee63ce46c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-3e8fd4d9-41bf-4b6e-9b00-89ee63ce46c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:39.244108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:39.245034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:39.247482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 161/313] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:39.255054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014b211-0d0d-443c-b97d-dba701830c9b None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.372336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014b211-0d0d-443c-b97d-dba701830c9b None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.409940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364485fb-1733-486d-b65e-8bb08d818da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.410556 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364485fb-1733-486d-b65e-8bb08d818da6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.410835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364485fb-1733-486d-b65e-8bb08d818da6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.411106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364485fb-1733-486d-b65e-8bb08d818da6 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.411810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364485fb-1733-486d-b65e-8bb08d818da6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:39.412110 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364485fb-1733-486d-b65e-8bb08d818da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.413494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364485fb-1733-486d-b65e-8bb08d818da6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.423815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 153/314] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:39.432256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fd4d9-41bf-4b6e-9b00-89ee63ce46c6 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.550784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fd4d9-41bf-4b6e-9b00-89ee63ce46c6 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.575548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.596806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba7c54-3fac-4da5-8846-5ecff4a0391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.597455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba7c54-3fac-4da5-8846-5ecff4a0391d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.597745 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba7c54-3fac-4da5-8846-5ecff4a0391d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.598017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba7c54-3fac-4da5-8846-5ecff4a0391d None admin] PATH_INFO: `/v3/users/621b6b3e24d8408da5aa083475c4c46b` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.599229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba7c54-3fac-4da5-8846-5ecff4a0391d None admin] RBAC: Authorizing `identity:get_user(user_id=621b6b3e24d8408da5aa083475c4c46b)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.600773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba7c54-3fac-4da5-8846-5ecff4a0391d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.602756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 162/315] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/users/621b6b3e24d8408da5aa083475c4c46b => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:39.609160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364485fb-1733-486d-b65e-8bb08d818da6 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.691244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.755794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364485fb-1733-486d-b65e-8bb08d818da6 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.793752 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13021f-0342-4391-9866-7c7252e24f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.794750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13021f-0342-4391-9866-7c7252e24f0b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.795287 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13021f-0342-4391-9866-7c7252e24f0b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.795774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13021f-0342-4391-9866-7c7252e24f0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.873995 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13021f-0342-4391-9866-7c7252e24f0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.875945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13021f-0342-4391-9866-7c7252e24f0b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.927698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.995646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 154/316] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:40.001467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba7c54-3fac-4da5-8846-5ecff4a0391d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.176557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba7c54-3fac-4da5-8846-5ecff4a0391d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.226794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10d810-3fe5-4187-8358-b57c26037e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.227540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10d810-3fe5-4187-8358-b57c26037e3d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.227822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10d810-3fe5-4187-8358-b57c26037e3d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.228097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10d810-3fe5-4187-8358-b57c26037e3d None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.229283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10d810-3fe5-4187-8358-b57c26037e3d None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.231718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10d810-3fe5-4187-8358-b57c26037e3d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.233762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 163/317] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:40.239791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13021f-0342-4391-9866-7c7252e24f0b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.373472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13021f-0342-4391-9866-7c7252e24f0b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.441960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d9284-1fa7-4631-8438-47cf4c3ce9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.442807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d9284-1fa7-4631-8438-47cf4c3ce9e3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.443346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d9284-1fa7-4631-8438-47cf4c3ce9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.443926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d9284-1fa7-4631-8438-47cf4c3ce9e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.445269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d9284-1fa7-4631-8438-47cf4c3ce9e3 None admin] RBAC: Adding query filter params (role.id=418096fb75b6406f8cb75452cb7b8edb, scope.project.id=7d8956a48042444687dd3ed3e39b1cd3, user.id=621b6b3e24d8408da5aa083475c4c46b) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:40.445836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d9284-1fa7-4631-8438-47cf4c3ce9e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.449162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d9284-1fa7-4631-8438-47cf4c3ce9e3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.461406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 155/318] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/role_assignments?user.id=621b6b3e24d8408da5aa083475c4c46b&scope.project.id=7d8956a48042444687dd3ed3e39b1cd3&role.id=418096fb75b6406f8cb75452cb7b8edb => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:40.487284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fea95b-9d24-474c-a687-1d4355f46acf None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.487622 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fea95b-9d24-474c-a687-1d4355f46acf None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.487855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fea95b-9d24-474c-a687-1d4355f46acf None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.489692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 164/319] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:40.495777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94edde-17ea-4ff1-9113-0fb8c48db454 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.496119 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94edde-17ea-4ff1-9113-0fb8c48db454 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.496334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94edde-17ea-4ff1-9113-0fb8c48db454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.517317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ac94edde-17ea-4ff1-9113-0fb8c48db454 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:40.518701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ac94edde-17ea-4ff1-9113-0fb8c48db454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:40.521256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:40.604275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 156/320] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:40.616347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d6aa3-74c6-48fb-95cc-de373c3ca2f1 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.616867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d6aa3-74c6-48fb-95cc-de373c3ca2f1 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.617373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d6aa3-74c6-48fb-95cc-de373c3ca2f1 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.621218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 165/321] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:40.630557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde05341-dd07-4fd0-b58d-f07e287e2517 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.630847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde05341-dd07-4fd0-b58d-f07e287e2517 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.631088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde05341-dd07-4fd0-b58d-f07e287e2517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.665574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-fde05341-dd07-4fd0-b58d-f07e287e2517 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:40.668032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fde05341-dd07-4fd0-b58d-f07e287e2517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:40.754925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eacfb-b37f-4548-b41f-ee7d1f49cf52 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.755657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eacfb-b37f-4548-b41f-ee7d1f49cf52 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.756136 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eacfb-b37f-4548-b41f-ee7d1f49cf52 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.757842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 166/322] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:40.765124 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bb2ef-6bea-47d0-9e6d-1416a82913a5 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.766004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bb2ef-6bea-47d0-9e6d-1416a82913a5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.766470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bb2ef-6bea-47d0-9e6d-1416a82913a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.788182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-842bb2ef-6bea-47d0-9e6d-1416a82913a5 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:40.789805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-842bb2ef-6bea-47d0-9e6d-1416a82913a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:40.802203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 157/323] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:40.882607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:40.888496 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde05341-dd07-4fd0-b58d-f07e287e2517 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.910536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 167/324] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:40.984829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.029661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde05341-dd07-4fd0-b58d-f07e287e2517 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.080572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bb2ef-6bea-47d0-9e6d-1416a82913a5 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.082209 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc50e9-7c86-4d27-9c5b-774314f715de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.083311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc50e9-7c86-4d27-9c5b-774314f715de None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.083791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc50e9-7c86-4d27-9c5b-774314f715de None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.084254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc50e9-7c86-4d27-9c5b-774314f715de None admin] PATH_INFO: `/v3/domains/default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.086163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc50e9-7c86-4d27-9c5b-774314f715de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.088845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc50e9-7c86-4d27-9c5b-774314f715de None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.092866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 158/325] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:41.101156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc50e9-7c86-4d27-9c5b-774314f715de None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.229969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc50e9-7c86-4d27-9c5b-774314f715de None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.263367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bb2ef-6bea-47d0-9e6d-1416a82913a5 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.270059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b066cce-df64-4ff0-bcc9-13cedabef675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.270679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b066cce-df64-4ff0-bcc9-13cedabef675 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.270957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b066cce-df64-4ff0-bcc9-13cedabef675 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.271271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b066cce-df64-4ff0-bcc9-13cedabef675 None admin] PATH_INFO: `/v3/groups` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.271910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b066cce-df64-4ff0-bcc9-13cedabef675 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.273358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b066cce-df64-4ff0-bcc9-13cedabef675 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.279867 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.294527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 159/326] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Aug 31 09:34:41 2022] POST /identity/v3/groups => generated 226 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:34:41.329780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8db5e6-152d-4e02-9112-478a483e043a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.330719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8db5e6-152d-4e02-9112-478a483e043a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.331181 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8db5e6-152d-4e02-9112-478a483e043a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.331613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8db5e6-152d-4e02-9112-478a483e043a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.346367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8db5e6-152d-4e02-9112-478a483e043a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-4e8db5e6-152d-4e02-9112-478a483e043a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:41.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:41.353685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:41.355211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:41.355211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:41.355211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:41.355211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:41.355211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:41.355211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:41.355211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:41.355211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:41.355211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:41.355211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:41.355211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:41.355211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:41.358734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 168/327] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:41.360737 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b066cce-df64-4ff0-bcc9-13cedabef675 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.427458 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8db5e6-152d-4e02-9112-478a483e043a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.515929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.570667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b066cce-df64-4ff0-bcc9-13cedabef675 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.620369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8db5e6-152d-4e02-9112-478a483e043a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.632536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26972b5f-337b-4d70-a6dd-6f61714015b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.633169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26972b5f-337b-4d70-a6dd-6f61714015b1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.633447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26972b5f-337b-4d70-a6dd-6f61714015b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.633716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26972b5f-337b-4d70-a6dd-6f61714015b1 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.634766 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26972b5f-337b-4d70-a6dd-6f61714015b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:41.634918 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26972b5f-337b-4d70-a6dd-6f61714015b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.637363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26972b5f-337b-4d70-a6dd-6f61714015b1 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.648197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 160/328] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:41.655087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26972b5f-337b-4d70-a6dd-6f61714015b1 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.676891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cca25-f63c-494f-82ff-a7ef6211eee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.678147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cca25-f63c-494f-82ff-a7ef6211eee6 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.678585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cca25-f63c-494f-82ff-a7ef6211eee6 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.679016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cca25-f63c-494f-82ff-a7ef6211eee6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.695732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cca25-f63c-494f-82ff-a7ef6211eee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-d30cca25-f63c-494f-82ff-a7ef6211eee6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:41.702482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:41.703465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:41.704242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:41.704242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:41.704242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:41.704242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:41.704242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:41.704242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:41.704242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:41.704242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:41.704242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:41.704242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:41.704242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:41.704242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:41.706847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 169/329] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:41.710286 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cca25-f63c-494f-82ff-a7ef6211eee6 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.750974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.833975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26972b5f-337b-4d70-a6dd-6f61714015b1 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.881068 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cca25-f63c-494f-82ff-a7ef6211eee6 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.882816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.891779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3dde-44c1-4d24-a5f8-5bf7f7b96b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.892442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3dde-44c1-4d24-a5f8-5bf7f7b96b10 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.892720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3dde-44c1-4d24-a5f8-5bf7f7b96b10 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.892994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3dde-44c1-4d24-a5f8-5bf7f7b96b10 None admin] PATH_INFO: `/v3/users/95fd141d4f704559878fe01dbf9878d5` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.895050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae3dde-44c1-4d24-a5f8-5bf7f7b96b10 None admin] RBAC: Authorizing `identity:get_user(user_id=95fd141d4f704559878fe01dbf9878d5)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.896595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae3dde-44c1-4d24-a5f8-5bf7f7b96b10 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.903156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 161/330] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/users/95fd141d4f704559878fe01dbf9878d5 => generated 293 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:41.908104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3dde-44c1-4d24-a5f8-5bf7f7b96b10 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.946992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c5250-6932-48c9-b912-a31d5baf5450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.947987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c5250-6932-48c9-b912-a31d5baf5450 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.948480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c5250-6932-48c9-b912-a31d5baf5450 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.948981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c5250-6932-48c9-b912-a31d5baf5450 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.950194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c5250-6932-48c9-b912-a31d5baf5450 None admin] RBAC: Adding query filter params (name=member) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:41.950703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c5250-6932-48c9-b912-a31d5baf5450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.953313 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c5250-6932-48c9-b912-a31d5baf5450 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.972010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 170/331] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:41.979097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c5250-6932-48c9-b912-a31d5baf5450 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:42.103597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3dde-44c1-4d24-a5f8-5bf7f7b96b10 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:42.157543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:42.175254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062aba8-eb4d-48c9-9a56-60538236d403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:42.176611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062aba8-eb4d-48c9-9a56-60538236d403 None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:42.177274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062aba8-eb4d-48c9-9a56-60538236d403 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:42.177929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062aba8-eb4d-48c9-9a56-60538236d403 None admin] PATH_INFO: `/v3/system/users/95fd141d4f704559878fe01dbf9878d5/roles/418096fb75b6406f8cb75452cb7b8edb` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:42.185424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062aba8-eb4d-48c9-9a56-60538236d403 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=95fd141d4f704559878fe01dbf9878d5, role_id=418096fb75b6406f8cb75452cb7b8edb)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:42.185782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c5250-6932-48c9-b912-a31d5baf5450 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:42.188569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062aba8-eb4d-48c9-9a56-60538236d403 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:42.203204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 162/332] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Aug 31 09:34:41 2022] PUT /identity/v3/system/users/95fd141d4f704559878fe01dbf9878d5/roles/418096fb75b6406f8cb75452cb7b8edb => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:34:42.273029 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3709b7-36a6-4e5d-95ad-05cdf1f721d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:42.274073 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3709b7-36a6-4e5d-95ad-05cdf1f721d1 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:42.274556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3709b7-36a6-4e5d-95ad-05cdf1f721d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:42.275030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3709b7-36a6-4e5d-95ad-05cdf1f721d1 None admin] PATH_INFO: `/v3/users/d55db7b2dbe240a7914b4f2ed34f74de` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:42.278324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3709b7-36a6-4e5d-95ad-05cdf1f721d1 None admin] RBAC: Authorizing `identity:get_user(user_id=d55db7b2dbe240a7914b4f2ed34f74de)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:42.280721 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3709b7-36a6-4e5d-95ad-05cdf1f721d1 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:42.284260 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 171/333] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/users/d55db7b2dbe240a7914b4f2ed34f74de => generated 293 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:42.290254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062aba8-eb4d-48c9-9a56-60538236d403 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:42.481342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062aba8-eb4d-48c9-9a56-60538236d403 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:42.522727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67608eeb-abd4-4a14-837a-5c80cd2131f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:42.523377 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67608eeb-abd4-4a14-837a-5c80cd2131f9 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:42.523653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67608eeb-abd4-4a14-837a-5c80cd2131f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:42.523925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67608eeb-abd4-4a14-837a-5c80cd2131f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:42.524667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67608eeb-abd4-4a14-837a-5c80cd2131f9 None admin] RBAC: Adding query filter params (role.id=7ccf7639a4864595b7bc589a9dc44df7, user.id=d55db7b2dbe240a7914b4f2ed34f74de, scope.system=all) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:42.524972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67608eeb-abd4-4a14-837a-5c80cd2131f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:42.526439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67608eeb-abd4-4a14-837a-5c80cd2131f9 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:42.547239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 163/334] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Aug 31 09:34:42 2022] GET /identity/v3/role_assignments?user.id=d55db7b2dbe240a7914b4f2ed34f74de&scope.system=all&role.id=7ccf7639a4864595b7bc589a9dc44df7 => generated 510 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:44.389737 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7decc2f-2ac0-4227-ba53-5ac52ec51559 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.390258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7decc2f-2ac0-4227-ba53-5ac52ec51559 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.390258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7decc2f-2ac0-4227-ba53-5ac52ec51559 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.392240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 172/335] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:44.399846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b5bc2-1f8b-468a-be71-1a75288ad97e None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.400353 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b5bc2-1f8b-468a-be71-1a75288ad97e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.400426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b5bc2-1f8b-468a-be71-1a75288ad97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.426235 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-796b5bc2-1f8b-468a-be71-1a75288ad97e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:44.427476 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-796b5bc2-1f8b-468a-be71-1a75288ad97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:44.490938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.530349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 164/336] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:44.780569 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7fa8c-7df4-4f29-8924-7e5dde92ee54 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.780968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7fa8c-7df4-4f29-8924-7e5dde92ee54 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.781144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7fa8c-7df4-4f29-8924-7e5dde92ee54 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.782796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 173/337] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:44.789078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902eb71-7f7d-4d13-9c44-e2c7c85cf3bc None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.789384 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902eb71-7f7d-4d13-9c44-e2c7c85cf3bc None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.789631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902eb71-7f7d-4d13-9c44-e2c7c85cf3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.800048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd21c04-3297-4ea1-a299-42a4d0ea58b5 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.800391 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd21c04-3297-4ea1-a299-42a4d0ea58b5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.800598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd21c04-3297-4ea1-a299-42a4d0ea58b5 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.802402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 174/338] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:44.809379 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe8195-61e4-4185-8080-43df5614e686 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.809379 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe8195-61e4-4185-8080-43df5614e686 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.809379 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe8195-61e4-4185-8080-43df5614e686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.826696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-e902eb71-7f7d-4d13-9c44-e2c7c85cf3bc None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:44.828009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e902eb71-7f7d-4d13-9c44-e2c7c85cf3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:44.844955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-85fe8195-61e4-4185-8080-43df5614e686 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:44.846201 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-85fe8195-61e4-4185-8080-43df5614e686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:44.918928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.990519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 175/339] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:44.998445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 165/340] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:45.369941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe8195-61e4-4185-8080-43df5614e686 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:45.383337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902eb71-7f7d-4d13-9c44-e2c7c85cf3bc None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:45.614155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe8195-61e4-4185-8080-43df5614e686 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:45.664157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902eb71-7f7d-4d13-9c44-e2c7c85cf3bc None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:45.676236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:45.696445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b5a6a-49e4-45c3-9720-6cf2bf8e57c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:45.697751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b5a6a-49e4-45c3-9720-6cf2bf8e57c5 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:45.698229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b5a6a-49e4-45c3-9720-6cf2bf8e57c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:45.698779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b5a6a-49e4-45c3-9720-6cf2bf8e57c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:45.705893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae174709-81c0-4dd8-865a-61e1b7c4f4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:45.706531 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae174709-81c0-4dd8-865a-61e1b7c4f4fe None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:45.706810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae174709-81c0-4dd8-865a-61e1b7c4f4fe None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:45.707094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae174709-81c0-4dd8-865a-61e1b7c4f4fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:45.719763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae174709-81c0-4dd8-865a-61e1b7c4f4fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:45.721402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b5a6a-49e4-45c3-9720-6cf2bf8e57c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-ae174709-81c0-4dd8-865a-61e1b7c4f4fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:45.724773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:45.726452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 166/341] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:45.729409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae174709-81c0-4dd8-865a-61e1b7c4f4fe None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-c86b5a6a-49e4-45c3-9720-6cf2bf8e57c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:45.732239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:45.738169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:45.739739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 176/342] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:45.747942 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b5a6a-49e4-45c3-9720-6cf2bf8e57c5 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:45.796517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:45.840010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae174709-81c0-4dd8-865a-61e1b7c4f4fe None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:45.877978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4030f-c0a2-450f-992c-59b9bdeadbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:45.878631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4030f-c0a2-450f-992c-59b9bdeadbdc None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:45.878909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4030f-c0a2-450f-992c-59b9bdeadbdc None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:45.879208 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4030f-c0a2-450f-992c-59b9bdeadbdc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:45.892009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4030f-c0a2-450f-992c-59b9bdeadbdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-8ee4030f-c0a2-450f-992c-59b9bdeadbdc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:45.897450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:45.899745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 167/343] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:45.901757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4030f-c0a2-450f-992c-59b9bdeadbdc None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:45.931521 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:45.962106 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b5a6a-49e4-45c3-9720-6cf2bf8e57c5 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:46.016899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4030f-c0a2-450f-992c-59b9bdeadbdc None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:46.059717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba6fa5-041e-429f-8349-a4b7de101e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:46.060913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b967f77-c162-4932-980f-442a641dd4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:46.061485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba6fa5-041e-429f-8349-a4b7de101e84 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.061635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b967f77-c162-4932-980f-442a641dd4ef None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.061913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b967f77-c162-4932-980f-442a641dd4ef None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.062165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b967f77-c162-4932-980f-442a641dd4ef None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.062844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b967f77-c162-4932-980f-442a641dd4ef None admin] RBAC: Adding query filter params (name=member) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:46.067161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b967f77-c162-4932-980f-442a641dd4ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:46.067161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b967f77-c162-4932-980f-442a641dd4ef None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:46.067880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba6fa5-041e-429f-8349-a4b7de101e84 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.068690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba6fa5-041e-429f-8349-a4b7de101e84 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.070388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba6fa5-041e-429f-8349-a4b7de101e84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:46.071247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba6fa5-041e-429f-8349-a4b7de101e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:46.074679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba6fa5-041e-429f-8349-a4b7de101e84 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:46.080354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 168/344] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:46.081097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b967f77-c162-4932-980f-442a641dd4ef None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:46.106547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 177/345] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:46.110200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba6fa5-041e-429f-8349-a4b7de101e84 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:46.185957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:46.196010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b967f77-c162-4932-980f-442a641dd4ef None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:46.237201 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8b40b-da47-4903-91bb-cc0a754aa4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:46.242116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8b40b-da47-4903-91bb-cc0a754aa4ef None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.242116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8b40b-da47-4903-91bb-cc0a754aa4ef None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.242116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8b40b-da47-4903-91bb-cc0a754aa4ef None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.242116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8b40b-da47-4903-91bb-cc0a754aa4ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:46.242116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8b40b-da47-4903-91bb-cc0a754aa4ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:46.242116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8b40b-da47-4903-91bb-cc0a754aa4ef None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:46.251202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 169/346] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:46.255382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8b40b-da47-4903-91bb-cc0a754aa4ef None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:46.356406 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:46.391603 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba6fa5-041e-429f-8349-a4b7de101e84 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:46.478255 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8b40b-da47-4903-91bb-cc0a754aa4ef None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:46.480468 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6592b-0f90-4868-9d4f-11743ac75d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:46.481270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6592b-0f90-4868-9d4f-11743ac75d9b None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.481559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6592b-0f90-4868-9d4f-11743ac75d9b None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.483193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6592b-0f90-4868-9d4f-11743ac75d9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.525845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca98ff0-616b-4699-af79-f774382ee454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:46.527074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca98ff0-616b-4699-af79-f774382ee454 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.529751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca98ff0-616b-4699-af79-f774382ee454 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.530103 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca98ff0-616b-4699-af79-f774382ee454 None admin] PATH_INFO: `/v3/users/95fd141d4f704559878fe01dbf9878d5` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.531519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca98ff0-616b-4699-af79-f774382ee454 None admin] RBAC: Authorizing `identity:get_user(user_id=95fd141d4f704559878fe01dbf9878d5)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:46.532997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca98ff0-616b-4699-af79-f774382ee454 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:46.535650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 170/347] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:46 2022] GET /identity/v3/users/95fd141d4f704559878fe01dbf9878d5 => generated 293 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:46.535963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca98ff0-616b-4699-af79-f774382ee454 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:46.575147 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:46.676742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6592b-0f90-4868-9d4f-11743ac75d9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:46.678267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6592b-0f90-4868-9d4f-11743ac75d9b None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:46.701764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca98ff0-616b-4699-af79-f774382ee454 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:46.769427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada1d6-b808-4208-9b90-d964b391ab35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:46.769953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada1d6-b808-4208-9b90-d964b391ab35 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.770234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada1d6-b808-4208-9b90-d964b391ab35 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.770507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada1d6-b808-4208-9b90-d964b391ab35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.835348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:46.854309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ada1d6-b808-4208-9b90-d964b391ab35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:46.856078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ada1d6-b808-4208-9b90-d964b391ab35 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:46.869344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 178/348] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:46.872457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c1736-bdf4-4d3a-a468-74995c55e45a None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.872961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c1736-bdf4-4d3a-a468-74995c55e45a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.873454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c1736-bdf4-4d3a-a468-74995c55e45a None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.876445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 179/349] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:46.877876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c1736-bdf4-4d3a-a468-74995c55e45a None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:46.879985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:46.981812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 171/350] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:46.983170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c35c-28cd-4411-a06b-077c12d131bc None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.983450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c35c-28cd-4411-a06b-077c12d131bc None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.983731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c35c-28cd-4411-a06b-077c12d131bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.004426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ec09c35c-28cd-4411-a06b-077c12d131bc None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:47.008061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ec09c35c-28cd-4411-a06b-077c12d131bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:47.027897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c1736-bdf4-4d3a-a468-74995c55e45a None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.069342 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687f543-8691-4673-a18a-41762ca083f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.069972 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687f543-8691-4673-a18a-41762ca083f8 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.070246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687f543-8691-4673-a18a-41762ca083f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.070516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687f543-8691-4673-a18a-41762ca083f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.071287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687f543-8691-4673-a18a-41762ca083f8 None admin] RBAC: Adding query filter params (role.id=418096fb75b6406f8cb75452cb7b8edb, user.id=95fd141d4f704559878fe01dbf9878d5, scope.system=all) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:47.071595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687f543-8691-4673-a18a-41762ca083f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.073103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687f543-8691-4673-a18a-41762ca083f8 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.090597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 180/351] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Aug 31 09:34:46 2022] GET /identity/v3/role_assignments?user.id=95fd141d4f704559878fe01dbf9878d5&scope.system=all&role.id=418096fb75b6406f8cb75452cb7b8edb => generated 510 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:47.091114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687f543-8691-4673-a18a-41762ca083f8 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.102684 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 172/352] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:47.103698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c35c-28cd-4411-a06b-077c12d131bc None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.152806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:47.174586 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:47.237600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687f543-8691-4673-a18a-41762ca083f8 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.265748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c35c-28cd-4411-a06b-077c12d131bc None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.317883 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf6f53-50d4-43c1-9fab-d23f8798c907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.318278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78637f0-78c9-4172-b0ec-d33533dab5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.318582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf6f53-50d4-43c1-9fab-d23f8798c907 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.318815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf6f53-50d4-43c1-9fab-d23f8798c907 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.319034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78637f0-78c9-4172-b0ec-d33533dab5aa None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.319287 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf6f53-50d4-43c1-9fab-d23f8798c907 None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.319544 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78637f0-78c9-4172-b0ec-d33533dab5aa None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.319772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78637f0-78c9-4172-b0ec-d33533dab5aa None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.322006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78637f0-78c9-4172-b0ec-d33533dab5aa None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.325226 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf6f53-50d4-43c1-9fab-d23f8798c907 None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.325728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78637f0-78c9-4172-b0ec-d33533dab5aa None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.326706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf6f53-50d4-43c1-9fab-d23f8798c907 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.328464 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 173/353] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:47.332230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 181/354] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:47.340518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf6f53-50d4-43c1-9fab-d23f8798c907 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.344632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78637f0-78c9-4172-b0ec-d33533dab5aa None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.500909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf6f53-50d4-43c1-9fab-d23f8798c907 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.526009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:47.551352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50659901-e646-4cfc-bab8-fca522f71c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.552011 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659901-e646-4cfc-bab8-fca522f71c9e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.552291 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659901-e646-4cfc-bab8-fca522f71c9e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.552528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659901-e646-4cfc-bab8-fca522f71c9e None admin] PATH_INFO: `/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.554585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78637f0-78c9-4172-b0ec-d33533dab5aa None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.560230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50659901-e646-4cfc-bab8-fca522f71c9e None admin] RBAC: Authorizing `identity:get_group(group_id=9662edef3cdd4b36b22b5a0921c8d3db)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.561597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50659901-e646-4cfc-bab8-fca522f71c9e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.563929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 174/355] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:47.586265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:47.588486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50659901-e646-4cfc-bab8-fca522f71c9e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.671950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544393b8-5532-47c1-91fb-07dedce38b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.672625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544393b8-5532-47c1-91fb-07dedce38b85 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.672909 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544393b8-5532-47c1-91fb-07dedce38b85 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.673185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544393b8-5532-47c1-91fb-07dedce38b85 None admin] PATH_INFO: `/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.674784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544393b8-5532-47c1-91fb-07dedce38b85 None admin] RBAC: Authorizing `identity:get_group(group_id=9662edef3cdd4b36b22b5a0921c8d3db)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.676985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544393b8-5532-47c1-91fb-07dedce38b85 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.679292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 182/356] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db => generated 226 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:47.680720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d0eb1-81d3-4fc1-bd8f-1b93e4483dfe None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.681012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d0eb1-81d3-4fc1-bd8f-1b93e4483dfe None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.681290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d0eb1-81d3-4fc1-bd8f-1b93e4483dfe None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.683469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 183/357] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:47.688399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d0eb1-81d3-4fc1-bd8f-1b93e4483dfe None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.761629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50659901-e646-4cfc-bab8-fca522f71c9e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.818606 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d30127-dcde-41a2-844e-45cb34a2e7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.819983 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d30127-dcde-41a2-844e-45cb34a2e7a3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.820455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d30127-dcde-41a2-844e-45cb34a2e7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.820911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d30127-dcde-41a2-844e-45cb34a2e7a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.822176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d30127-dcde-41a2-844e-45cb34a2e7a3 None admin] RBAC: Adding query filter params (group.id=9662edef3cdd4b36b22b5a0921c8d3db, role.id=7ccf7639a4864595b7bc589a9dc44df7, scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:47.822714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d30127-dcde-41a2-844e-45cb34a2e7a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.825743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d30127-dcde-41a2-844e-45cb34a2e7a3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.833242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 175/358] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/role_assignments?group.id=9662edef3cdd4b36b22b5a0921c8d3db&scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2&role.id=7ccf7639a4864595b7bc589a9dc44df7 => generated 260 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:47.834788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79473ab-b9c5-4167-bd66-3fd05733b788 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.835299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79473ab-b9c5-4167-bd66-3fd05733b788 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.835761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79473ab-b9c5-4167-bd66-3fd05733b788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.858963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-e79473ab-b9c5-4167-bd66-3fd05733b788 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:47.860665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e79473ab-b9c5-4167-bd66-3fd05733b788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:47.867897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d0eb1-81d3-4fc1-bd8f-1b93e4483dfe None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.927726 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519b5cf-4e00-48b5-9931-de232ed48c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.928711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519b5cf-4e00-48b5-9931-de232ed48c46 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.929233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519b5cf-4e00-48b5-9931-de232ed48c46 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.929699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519b5cf-4e00-48b5-9931-de232ed48c46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.930673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519b5cf-4e00-48b5-9931-de232ed48c46 None admin] RBAC: Adding query filter params (group.id=9662edef3cdd4b36b22b5a0921c8d3db, role.id=83a1e8de33194004a2a672104465c819, scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:47.931234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519b5cf-4e00-48b5-9931-de232ed48c46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.932196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:47.933144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519b5cf-4e00-48b5-9931-de232ed48c46 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.941022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 184/359] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/role_assignments?group.id=9662edef3cdd4b36b22b5a0921c8d3db&scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2&role.id=83a1e8de33194004a2a672104465c819 => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:47.942615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2937ea7-7f67-4691-993e-7eb018473162 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.942902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2937ea7-7f67-4691-993e-7eb018473162 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.943209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2937ea7-7f67-4691-993e-7eb018473162 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.945599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 185/360] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:47.946511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2937ea7-7f67-4691-993e-7eb018473162 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.994162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 176/361] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:47.997565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa5a1a-0c81-4e9d-9406-3ddea07c7709 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.997855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa5a1a-0c81-4e9d-9406-3ddea07c7709 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.998242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa5a1a-0c81-4e9d-9406-3ddea07c7709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.020215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:48.025843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a6fa5a1a-0c81-4e9d-9406-3ddea07c7709 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:48.028774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a6fa5a1a-0c81-4e9d-9406-3ddea07c7709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:48.134093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2937ea7-7f67-4691-993e-7eb018473162 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:48.154273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 177/362] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:48.191808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c433f2e-c352-45cc-858e-c56c5b2c1440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:48.192809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c433f2e-c352-45cc-858e-c56c5b2c1440 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.193294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c433f2e-c352-45cc-858e-c56c5b2c1440 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.193766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c433f2e-c352-45cc-858e-c56c5b2c1440 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.208316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c433f2e-c352-45cc-858e-c56c5b2c1440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-5c433f2e-c352-45cc-858e-c56c5b2c1440 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:48.214552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:48.215590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:48.216390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 186/363] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:48.221826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa5a1a-0c81-4e9d-9406-3ddea07c7709 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:48.337770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c433f2e-c352-45cc-858e-c56c5b2c1440 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:48.397020 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa5a1a-0c81-4e9d-9406-3ddea07c7709 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:48.420123 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:48.448228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee311475-cc19-4d6c-ac1d-cc188344baed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:48.449756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee311475-cc19-4d6c-ac1d-cc188344baed None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.450156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee311475-cc19-4d6c-ac1d-cc188344baed None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.450431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee311475-cc19-4d6c-ac1d-cc188344baed None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.451112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee311475-cc19-4d6c-ac1d-cc188344baed None admin] RBAC: Adding query filter params (name=member) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:48.451438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee311475-cc19-4d6c-ac1d-cc188344baed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:48.452833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee311475-cc19-4d6c-ac1d-cc188344baed None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:48.476086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 178/364] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:48.482148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee311475-cc19-4d6c-ac1d-cc188344baed None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:48.546143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c433f2e-c352-45cc-858e-c56c5b2c1440 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:48.549436 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:48.608933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844c2fe-a2cc-4bfa-897f-6ed0822c2a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:48.610017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844c2fe-a2cc-4bfa-897f-6ed0822c2a21 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.610519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844c2fe-a2cc-4bfa-897f-6ed0822c2a21 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.610992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844c2fe-a2cc-4bfa-897f-6ed0822c2a21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.612784 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee311475-cc19-4d6c-ac1d-cc188344baed None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:48.638336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844c2fe-a2cc-4bfa-897f-6ed0822c2a21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-4844c2fe-a2cc-4bfa-897f-6ed0822c2a21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:48.646356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:48.647401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:48.648247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:48.648247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:48.648247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:48.648247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:48.648247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:48.648247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:48.648247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:48.648247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:48.648247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:48.648247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:48.648247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:48.648247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:48.649092 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 187/365] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:48.653516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844c2fe-a2cc-4bfa-897f-6ed0822c2a21 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:48.675556 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78599b2a-38a5-4d70-a01a-90dcfc9cd726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:48.676280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78599b2a-38a5-4d70-a01a-90dcfc9cd726 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.676630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78599b2a-38a5-4d70-a01a-90dcfc9cd726 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.676989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78599b2a-38a5-4d70-a01a-90dcfc9cd726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.723803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:48.768911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78599b2a-38a5-4d70-a01a-90dcfc9cd726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:48.770399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78599b2a-38a5-4d70-a01a-90dcfc9cd726 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:48.811774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844c2fe-a2cc-4bfa-897f-6ed0822c2a21 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:48.862435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c19b7-f288-4456-87cb-1f3a2726e247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:48.863485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c19b7-f288-4456-87cb-1f3a2726e247 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.864489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c19b7-f288-4456-87cb-1f3a2726e247 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.865022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c19b7-f288-4456-87cb-1f3a2726e247 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.881299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c19b7-f288-4456-87cb-1f3a2726e247 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-0c2c19b7-f288-4456-87cb-1f3a2726e247 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:48.888430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:48.889371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:48.890131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:48.890131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:48.890131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:48.890131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:48.890131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:48.890131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:48.890131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:48.890131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:48.890131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:48.890131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:48.890131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:48.890131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:48.892668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 188/366] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:48.893323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c19b7-f288-4456-87cb-1f3a2726e247 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:48.905900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 179/367] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:48.906704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78599b2a-38a5-4d70-a01a-90dcfc9cd726 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:48.929188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.006412 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.048532 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c19b7-f288-4456-87cb-1f3a2726e247 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.057844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78599b2a-38a5-4d70-a01a-90dcfc9cd726 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.101700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754eff3-9e07-40cd-93ca-67143e130e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.103303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754eff3-9e07-40cd-93ca-67143e130e6c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.103817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754eff3-9e07-40cd-93ca-67143e130e6c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.104312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754eff3-9e07-40cd-93ca-67143e130e6c None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.107025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754eff3-9e07-40cd-93ca-67143e130e6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:49.107600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754eff3-9e07-40cd-93ca-67143e130e6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.109521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754eff3-9e07-40cd-93ca-67143e130e6c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.126466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 189/368] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:49.128097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754eff3-9e07-40cd-93ca-67143e130e6c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.128621 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bd056-a578-4795-8030-5bb26099a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.129679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bd056-a578-4795-8030-5bb26099a836 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.130095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bd056-a578-4795-8030-5bb26099a836 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.130375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bd056-a578-4795-8030-5bb26099a836 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.131071 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bd056-a578-4795-8030-5bb26099a836 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:49.132381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bd056-a578-4795-8030-5bb26099a836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.134599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bd056-a578-4795-8030-5bb26099a836 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.153448 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 180/369] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:49.155234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bd056-a578-4795-8030-5bb26099a836 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.258625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.281466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754eff3-9e07-40cd-93ca-67143e130e6c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.321779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8d6be-60f4-43da-aa20-ea86f9402571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.322786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8d6be-60f4-43da-aa20-ea86f9402571 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.324150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8d6be-60f4-43da-aa20-ea86f9402571 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.324935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8d6be-60f4-43da-aa20-ea86f9402571 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.326495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8d6be-60f4-43da-aa20-ea86f9402571 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.328214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8d6be-60f4-43da-aa20-ea86f9402571 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.330220 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 190/370] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:49.331604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8d6be-60f4-43da-aa20-ea86f9402571 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.357704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bd056-a578-4795-8030-5bb26099a836 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.401969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.412644 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e2395-9d0c-4e8d-bbbc-4816c4be6d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.413589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e2395-9d0c-4e8d-bbbc-4816c4be6d45 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.414069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e2395-9d0c-4e8d-bbbc-4816c4be6d45 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.414357 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e2395-9d0c-4e8d-bbbc-4816c4be6d45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.457239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8d6be-60f4-43da-aa20-ea86f9402571 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.514285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c51dc-0513-45d9-bdb9-5c28942928dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.515700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c51dc-0513-45d9-bdb9-5c28942928dd None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.516335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c51dc-0513-45d9-bdb9-5c28942928dd None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.517032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c51dc-0513-45d9-bdb9-5c28942928dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.546266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e2395-9d0c-4e8d-bbbc-4816c4be6d45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.547826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e2395-9d0c-4e8d-bbbc-4816c4be6d45 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.556706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.610738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c51dc-0513-45d9-bdb9-5c28942928dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.612445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c51dc-0513-45d9-bdb9-5c28942928dd None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.677589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 181/371] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:49.678788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e2395-9d0c-4e8d-bbbc-4816c4be6d45 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.721441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.740968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 191/372] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:49.742086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c51dc-0513-45d9-bdb9-5c28942928dd None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.806270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.816974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e2395-9d0c-4e8d-bbbc-4816c4be6d45 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.864235 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c45ac9-dfa1-425a-82cc-431af2dbfc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.865212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c45ac9-dfa1-425a-82cc-431af2dbfc8c None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.865658 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c45ac9-dfa1-425a-82cc-431af2dbfc8c None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.866090 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c45ac9-dfa1-425a-82cc-431af2dbfc8c None admin] PATH_INFO: `/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.866256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c51dc-0513-45d9-bdb9-5c28942928dd None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.868228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c45ac9-dfa1-425a-82cc-431af2dbfc8c None admin] RBAC: Authorizing `identity:get_group(group_id=9662edef3cdd4b36b22b5a0921c8d3db)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.870689 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c45ac9-dfa1-425a-82cc-431af2dbfc8c None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.873211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 182/373] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:49.874592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c45ac9-dfa1-425a-82cc-431af2dbfc8c None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.911049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841a1f3-93a1-47f6-9061-0714d2feea91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.911890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841a1f3-93a1-47f6-9061-0714d2feea91 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.912272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841a1f3-93a1-47f6-9061-0714d2feea91 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.912647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841a1f3-93a1-47f6-9061-0714d2feea91 None admin] PATH_INFO: `/v3/projects/d473eacf42fc47c2aa82e80873c7637e` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.914009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841a1f3-93a1-47f6-9061-0714d2feea91 None admin] RBAC: Authorizing `identity:get_project(project_id=d473eacf42fc47c2aa82e80873c7637e)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.915606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841a1f3-93a1-47f6-9061-0714d2feea91 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.922723 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 192/374] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/projects/d473eacf42fc47c2aa82e80873c7637e => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:49.923877 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841a1f3-93a1-47f6-9061-0714d2feea91 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.041863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841a1f3-93a1-47f6-9061-0714d2feea91 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.057049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:50.062822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c45ac9-dfa1-425a-82cc-431af2dbfc8c None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.067369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:50.087330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc930616-a1b0-4703-b8c7-9864b1b16dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.088044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc930616-a1b0-4703-b8c7-9864b1b16dbe None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.088448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc930616-a1b0-4703-b8c7-9864b1b16dbe None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.088781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc930616-a1b0-4703-b8c7-9864b1b16dbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.090359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc930616-a1b0-4703-b8c7-9864b1b16dbe None admin] RBAC: Adding query filter params (group.id=9662edef3cdd4b36b22b5a0921c8d3db, role.id=7ccf7639a4864595b7bc589a9dc44df7, scope.project.id=7d8956a48042444687dd3ed3e39b1cd3) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:50.090834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc930616-a1b0-4703-b8c7-9864b1b16dbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.092336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc930616-a1b0-4703-b8c7-9864b1b16dbe None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.098644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 193/375] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/role_assignments?group.id=9662edef3cdd4b36b22b5a0921c8d3db&scope.project.id=7d8956a48042444687dd3ed3e39b1cd3&role.id=7ccf7639a4864595b7bc589a9dc44df7 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.099622 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc930616-a1b0-4703-b8c7-9864b1b16dbe None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.110493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d94b12-6fcb-49ff-8e68-46e598c18db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.111317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d94b12-6fcb-49ff-8e68-46e598c18db9 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.111599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d94b12-6fcb-49ff-8e68-46e598c18db9 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.111871 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d94b12-6fcb-49ff-8e68-46e598c18db9 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.115382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d94b12-6fcb-49ff-8e68-46e598c18db9 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.117144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d94b12-6fcb-49ff-8e68-46e598c18db9 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.119549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 183/376] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.127437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d94b12-6fcb-49ff-8e68-46e598c18db9 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.227492 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc930616-a1b0-4703-b8c7-9864b1b16dbe None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.257344 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d94b12-6fcb-49ff-8e68-46e598c18db9 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.271955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafd68b-006b-48a2-bf8e-21a2516fbfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.272617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafd68b-006b-48a2-bf8e-21a2516fbfe9 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.272899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafd68b-006b-48a2-bf8e-21a2516fbfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.273171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafd68b-006b-48a2-bf8e-21a2516fbfe9 None admin] PATH_INFO: `/v3/groups/04ed80dd97d24094b4d269b4599608d1` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.281257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafd68b-006b-48a2-bf8e-21a2516fbfe9 None admin] RBAC: Authorizing `identity:get_group(group_id=04ed80dd97d24094b4d269b4599608d1)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.282770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafd68b-006b-48a2-bf8e-21a2516fbfe9 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.289731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 194/377] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/groups/04ed80dd97d24094b4d269b4599608d1 => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.291201 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafd68b-006b-48a2-bf8e-21a2516fbfe9 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.301930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c0863-f1b9-4fa6-bf28-85b1bf4f0feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.302588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c0863-f1b9-4fa6-bf28-85b1bf4f0feb None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.302920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c0863-f1b9-4fa6-bf28-85b1bf4f0feb None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.303224 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c0863-f1b9-4fa6-bf28-85b1bf4f0feb None admin] PATH_INFO: `/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.304508 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c0863-f1b9-4fa6-bf28-85b1bf4f0feb None admin] RBAC: Authorizing `identity:get_group(group_id=9662edef3cdd4b36b22b5a0921c8d3db)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.306065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c0863-f1b9-4fa6-bf28-85b1bf4f0feb None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.308551 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 184/378] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.317614 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c0863-f1b9-4fa6-bf28-85b1bf4f0feb None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.352166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:50.361170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:50.419626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafd68b-006b-48a2-bf8e-21a2516fbfe9 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.445174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c0863-f1b9-4fa6-bf28-85b1bf4f0feb None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.461395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3076d-0627-4303-b1bf-374d8a4cf69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.462363 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3076d-0627-4303-b1bf-374d8a4cf69c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.462790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3076d-0627-4303-b1bf-374d8a4cf69c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.463232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3076d-0627-4303-b1bf-374d8a4cf69c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.464789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3076d-0627-4303-b1bf-374d8a4cf69c None admin] RBAC: Adding query filter params (group.id=04ed80dd97d24094b4d269b4599608d1, role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=d473eacf42fc47c2aa82e80873c7637e) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:50.465537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3076d-0627-4303-b1bf-374d8a4cf69c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.467231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3076d-0627-4303-b1bf-374d8a4cf69c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.479272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 195/379] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/role_assignments?group.id=04ed80dd97d24094b4d269b4599608d1&scope.project.id=d473eacf42fc47c2aa82e80873c7637e&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.489283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99f6a6-2ee1-4e21-938e-42531000977c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.489922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99f6a6-2ee1-4e21-938e-42531000977c None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.490256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99f6a6-2ee1-4e21-938e-42531000977c None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.490536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99f6a6-2ee1-4e21-938e-42531000977c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.491365 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99f6a6-2ee1-4e21-938e-42531000977c None admin] RBAC: Adding query filter params (group.id=9662edef3cdd4b36b22b5a0921c8d3db, role.id=83a1e8de33194004a2a672104465c819, scope.project.id=7d8956a48042444687dd3ed3e39b1cd3) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:50.491679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99f6a6-2ee1-4e21-938e-42531000977c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.493291 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99f6a6-2ee1-4e21-938e-42531000977c None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.501638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 185/380] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/role_assignments?group.id=9662edef3cdd4b36b22b5a0921c8d3db&scope.project.id=7d8956a48042444687dd3ed3e39b1cd3&role.id=83a1e8de33194004a2a672104465c819 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.642163 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dea4d-b87d-402e-a369-b758c1cc64c6 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.642460 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dea4d-b87d-402e-a369-b758c1cc64c6 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.642621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dea4d-b87d-402e-a369-b758c1cc64c6 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.644850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 196/381] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:50.650985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca7c74-a177-4bd9-9f8c-08932f949f9a None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.651356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca7c74-a177-4bd9-9f8c-08932f949f9a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.652322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca7c74-a177-4bd9-9f8c-08932f949f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.677321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-adca7c74-a177-4bd9-9f8c-08932f949f9a None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:50.679077 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-adca7c74-a177-4bd9-9f8c-08932f949f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:50.776660 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 186/382] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:50.864623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ff21-777d-4099-8dcc-bc419dfe0685 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.865269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ff21-777d-4099-8dcc-bc419dfe0685 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.865743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ff21-777d-4099-8dcc-bc419dfe0685 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.868036 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 197/383] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:50.873582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355445ca-db74-4d79-a0d0-5df07049cb72 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.874557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355445ca-db74-4d79-a0d0-5df07049cb72 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.875028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355445ca-db74-4d79-a0d0-5df07049cb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.901550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-355445ca-db74-4d79-a0d0-5df07049cb72 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:50.904088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-355445ca-db74-4d79-a0d0-5df07049cb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:50.937545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:51.001990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 187/384] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:51.177052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2ff21-777d-4099-8dcc-bc419dfe0685 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:51.355379 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2ff21-777d-4099-8dcc-bc419dfe0685 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:51.391697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:51.438634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3c509-119c-4fae-8bfa-1699e7cbea09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:51.439464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3c509-119c-4fae-8bfa-1699e7cbea09 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:51.439894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3c509-119c-4fae-8bfa-1699e7cbea09 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:51.440335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3c509-119c-4fae-8bfa-1699e7cbea09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:51.474579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3c509-119c-4fae-8bfa-1699e7cbea09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-ddb3c509-119c-4fae-8bfa-1699e7cbea09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:51.484983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:51.485948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:51.486740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:51.486740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:51.486740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:51.486740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:51.486740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:51.486740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:51.486740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:51.486740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:51.486740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:51.486740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:51.486740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:51.486740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:51.487365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 198/385] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:51.496926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355445ca-db74-4d79-a0d0-5df07049cb72 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:51.615461 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3c509-119c-4fae-8bfa-1699e7cbea09 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:51.679098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355445ca-db74-4d79-a0d0-5df07049cb72 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:51.744131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3c509-119c-4fae-8bfa-1699e7cbea09 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:51.755101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c34aa-2cd1-4462-9399-fd45316c883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:51.756098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c34aa-2cd1-4462-9399-fd45316c883b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:51.756614 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c34aa-2cd1-4462-9399-fd45316c883b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:51.757108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c34aa-2cd1-4462-9399-fd45316c883b None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:51.757986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c34aa-2cd1-4462-9399-fd45316c883b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:51.758501 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c34aa-2cd1-4462-9399-fd45316c883b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:51.760137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c34aa-2cd1-4462-9399-fd45316c883b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:51.780068 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 188/386] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:51.795752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43bad2-ddad-4ed1-a37c-cebdfa51a8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:51.796403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43bad2-ddad-4ed1-a37c-cebdfa51a8ed None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:51.796683 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43bad2-ddad-4ed1-a37c-cebdfa51a8ed None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:51.796957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43bad2-ddad-4ed1-a37c-cebdfa51a8ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:51.801017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c34aa-2cd1-4462-9399-fd45316c883b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:51.813355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43bad2-ddad-4ed1-a37c-cebdfa51a8ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-7a43bad2-ddad-4ed1-a37c-cebdfa51a8ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:51.818992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:51.820044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:51.820818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:51.820818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:51.820818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:51.820818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:51.820818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:51.820818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:51.820818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:51.820818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:51.820818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:51.820818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:51.820818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:51.820818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:51.831166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 199/387] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:51.831166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43bad2-ddad-4ed1-a37c-cebdfa51a8ed None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:51.852677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:51.870745 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:51.955611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43bad2-ddad-4ed1-a37c-cebdfa51a8ed None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:51.999430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbca92c-c054-46d2-999f-f5b2f4b2c8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:51.999973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbca92c-c054-46d2-999f-f5b2f4b2c8b6 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.000252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbca92c-c054-46d2-999f-f5b2f4b2c8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.000524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbca92c-c054-46d2-999f-f5b2f4b2c8b6 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.001207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbca92c-c054-46d2-999f-f5b2f4b2c8b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:52.001510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbca92c-c054-46d2-999f-f5b2f4b2c8b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:52.002987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbca92c-c054-46d2-999f-f5b2f4b2c8b6 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:52.017221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 200/388] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:52.020023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbca92c-c054-46d2-999f-f5b2f4b2c8b6 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:52.105458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c34aa-2cd1-4462-9399-fd45316c883b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:52.139758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:52.149095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbca92c-c054-46d2-999f-f5b2f4b2c8b6 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:52.195033 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b43075-8bb9-4964-8582-c3be658ba533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:52.195711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b43075-8bb9-4964-8582-c3be658ba533 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.195989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b43075-8bb9-4964-8582-c3be658ba533 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.196261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b43075-8bb9-4964-8582-c3be658ba533 None admin] PATH_INFO: `/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.197300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74e3c3-2589-456f-84f2-bbbae424bb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:52.197520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b43075-8bb9-4964-8582-c3be658ba533 None admin] RBAC: Authorizing `identity:get_group(group_id=9662edef3cdd4b36b22b5a0921c8d3db)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:52.198413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74e3c3-2589-456f-84f2-bbbae424bb28 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.198899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74e3c3-2589-456f-84f2-bbbae424bb28 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.199510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74e3c3-2589-456f-84f2-bbbae424bb28 None admin] PATH_INFO: `/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.199757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b43075-8bb9-4964-8582-c3be658ba533 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:52.201980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74e3c3-2589-456f-84f2-bbbae424bb28 None admin] RBAC: Authorizing `identity:get_group(group_id=9662edef3cdd4b36b22b5a0921c8d3db)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:52.203773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74e3c3-2589-456f-84f2-bbbae424bb28 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:52.203894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 189/389] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:51 2022] GET /identity/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db => generated 226 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:52.206110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 201/390] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:52 2022] GET /identity/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:52.210161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b43075-8bb9-4964-8582-c3be658ba533 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:52.220194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74e3c3-2589-456f-84f2-bbbae424bb28 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:52.391240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74e3c3-2589-456f-84f2-bbbae424bb28 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:52.419753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:52.440330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2972969-44bf-49e9-8e17-b83ba0c9afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:52.441351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2972969-44bf-49e9-8e17-b83ba0c9afde None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.441873 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2972969-44bf-49e9-8e17-b83ba0c9afde None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.442353 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2972969-44bf-49e9-8e17-b83ba0c9afde None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.443866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2972969-44bf-49e9-8e17-b83ba0c9afde None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:52.445559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2972969-44bf-49e9-8e17-b83ba0c9afde None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:52.446541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b43075-8bb9-4964-8582-c3be658ba533 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:52.448036 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 202/391] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:52 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:52.493079 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2972969-44bf-49e9-8e17-b83ba0c9afde None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:52.541388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a33c0-d7a5-4b6b-8e1d-d8e51568535c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:52.543971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a33c0-d7a5-4b6b-8e1d-d8e51568535c None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.544466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a33c0-d7a5-4b6b-8e1d-d8e51568535c None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.544947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a33c0-d7a5-4b6b-8e1d-d8e51568535c None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.548789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:52.550590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a33c0-d7a5-4b6b-8e1d-d8e51568535c None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:52.553711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a33c0-d7a5-4b6b-8e1d-d8e51568535c None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:52.564134 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 190/392] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:52 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:52.570694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a33c0-d7a5-4b6b-8e1d-d8e51568535c None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:52.722693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2972969-44bf-49e9-8e17-b83ba0c9afde None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:52.785095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bc9a5-dd10-4d33-a699-25b58eaed6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:52.785967 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bc9a5-dd10-4d33-a699-25b58eaed6ee None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.786352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bc9a5-dd10-4d33-a699-25b58eaed6ee None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.786732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bc9a5-dd10-4d33-a699-25b58eaed6ee None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2/groups/9662edef3cdd4b36b22b5a0921c8d3db/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.791439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bc9a5-dd10-4d33-a699-25b58eaed6ee None admin] RBAC: Authorizing `identity:create_grant(project_id=76ea9635caef4c68bc934ba9f9ee98f2, group_id=9662edef3cdd4b36b22b5a0921c8d3db, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:52.793767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bc9a5-dd10-4d33-a699-25b58eaed6ee None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:52.830586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 203/393] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Aug 31 09:34:52 2022] PUT /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2/groups/9662edef3cdd4b36b22b5a0921c8d3db/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:34:52.831556 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a33c0-d7a5-4b6b-8e1d-d8e51568535c None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:52.873711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdf948-f593-466f-9bb4-ef3f38fbca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:52.874711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdf948-f593-466f-9bb4-ef3f38fbca81 None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.875164 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdf948-f593-466f-9bb4-ef3f38fbca81 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.875581 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdf948-f593-466f-9bb4-ef3f38fbca81 None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2/groups/9662edef3cdd4b36b22b5a0921c8d3db/roles/83a1e8de33194004a2a672104465c819` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.878079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdf948-f593-466f-9bb4-ef3f38fbca81 None admin] RBAC: Authorizing `identity:create_grant(project_id=76ea9635caef4c68bc934ba9f9ee98f2, group_id=9662edef3cdd4b36b22b5a0921c8d3db, role_id=83a1e8de33194004a2a672104465c819)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:52.879992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdf948-f593-466f-9bb4-ef3f38fbca81 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:52.897738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 191/394] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Aug 31 09:34:52 2022] PUT /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2/groups/9662edef3cdd4b36b22b5a0921c8d3db/roles/83a1e8de33194004a2a672104465c819 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:53.379531 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54df022-8b94-4e29-b131-56f88bdc5a46 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.380581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54df022-8b94-4e29-b131-56f88bdc5a46 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.381113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54df022-8b94-4e29-b131-56f88bdc5a46 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.383103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 204/395] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:53.389099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f7dd0-1681-448d-81ec-e15d931f11fe None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.389604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f7dd0-1681-448d-81ec-e15d931f11fe None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.389702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f7dd0-1681-448d-81ec-e15d931f11fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.415830 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-5b1f7dd0-1681-448d-81ec-e15d931f11fe None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:53.417658 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5b1f7dd0-1681-448d-81ec-e15d931f11fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:53.436773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:53.516822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 192/396] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:53.968091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d0724-8377-434d-80be-25dea456d198 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.968378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d0724-8377-434d-80be-25dea456d198 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.968911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d0724-8377-434d-80be-25dea456d198 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.970860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 205/397] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:53.978930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c05009-ec9a-43ea-9550-6de711cdb47e None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.979262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c05009-ec9a-43ea-9550-6de711cdb47e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.979771 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c05009-ec9a-43ea-9550-6de711cdb47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.005102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-57c05009-ec9a-43ea-9550-6de711cdb47e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:54.006669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-57c05009-ec9a-43ea-9550-6de711cdb47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:54.081070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d0724-8377-434d-80be-25dea456d198 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:54.139074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 193/398] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:54.140774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b16b6-23de-4133-8f5b-fe709fe7d748 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.141061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b16b6-23de-4133-8f5b-fe709fe7d748 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.141334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b16b6-23de-4133-8f5b-fe709fe7d748 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.146768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 194/399] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:54.148464 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83588bf-9f03-410f-82c7-1794e9b59921 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.148750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83588bf-9f03-410f-82c7-1794e9b59921 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.149021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83588bf-9f03-410f-82c7-1794e9b59921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.170806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a83588bf-9f03-410f-82c7-1794e9b59921 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:54.173202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a83588bf-9f03-410f-82c7-1794e9b59921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:54.390104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:54.414847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 195/400] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:54.427164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d0724-8377-434d-80be-25dea456d198 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:54.481721 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47e76f-f8f0-4fd7-bb92-c128a60311ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:54.482396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47e76f-f8f0-4fd7-bb92-c128a60311ae None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.482680 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47e76f-f8f0-4fd7-bb92-c128a60311ae None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.482959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47e76f-f8f0-4fd7-bb92-c128a60311ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.506036 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec47e76f-f8f0-4fd7-bb92-c128a60311ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-ec47e76f-f8f0-4fd7-bb92-c128a60311ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:54.511771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:54.513495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:54.515021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 206/401] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:54.523385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83588bf-9f03-410f-82c7-1794e9b59921 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:54.543592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:54.611345 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47e76f-f8f0-4fd7-bb92-c128a60311ae None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:54.742410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83588bf-9f03-410f-82c7-1794e9b59921 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:54.790717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3ceca-d75a-4f4f-ad21-4a07e9478c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:54.792068 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3ceca-d75a-4f4f-ad21-4a07e9478c11 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.792819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3ceca-d75a-4f4f-ad21-4a07e9478c11 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.793426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3ceca-d75a-4f4f-ad21-4a07e9478c11 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.794337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3ceca-d75a-4f4f-ad21-4a07e9478c11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:54.794879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3ceca-d75a-4f4f-ad21-4a07e9478c11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:54.796526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3ceca-d75a-4f4f-ad21-4a07e9478c11 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:54.805115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47e76f-f8f0-4fd7-bb92-c128a60311ae None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:54.816194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 196/402] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:54.818144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3ceca-d75a-4f4f-ad21-4a07e9478c11 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:54.859416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8a196-aea0-4f34-afe3-da876b3ae3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:54.860310 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8a196-aea0-4f34-afe3-da876b3ae3c7 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.860736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8a196-aea0-4f34-afe3-da876b3ae3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.861183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8a196-aea0-4f34-afe3-da876b3ae3c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.875457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8a196-aea0-4f34-afe3-da876b3ae3c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-82b8a196-aea0-4f34-afe3-da876b3ae3c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:54.881395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:54.882338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:54.883149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:54.883149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:54.883149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:54.883149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:54.883149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:54.883149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:54.883149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:54.883149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:54.883149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:54.883149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:54.883149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:54.883149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:54.886693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 207/403] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:54.888468 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8a196-aea0-4f34-afe3-da876b3ae3c7 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:54.910904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:54.973164 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:55.015605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8a196-aea0-4f34-afe3-da876b3ae3c7 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.049028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3ceca-d75a-4f4f-ad21-4a07e9478c11 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.057272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8785ce2-9dd6-4c48-8c1d-e47cfbcdc6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.058876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8785ce2-9dd6-4c48-8c1d-e47cfbcdc6d9 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.059431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8785ce2-9dd6-4c48-8c1d-e47cfbcdc6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.059914 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8785ce2-9dd6-4c48-8c1d-e47cfbcdc6d9 None admin] PATH_INFO: `/v3/groups/04ed80dd97d24094b4d269b4599608d1` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.061399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8785ce2-9dd6-4c48-8c1d-e47cfbcdc6d9 None admin] RBAC: Authorizing `identity:get_group(group_id=04ed80dd97d24094b4d269b4599608d1)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.063070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8785ce2-9dd6-4c48-8c1d-e47cfbcdc6d9 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.065009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 208/404] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:54 2022] GET /identity/v3/groups/04ed80dd97d24094b4d269b4599608d1 => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:55.066001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8785ce2-9dd6-4c48-8c1d-e47cfbcdc6d9 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.119557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af152522-33c1-409f-896d-1fd19861e72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.121661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af152522-33c1-409f-896d-1fd19861e72e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.122520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af152522-33c1-409f-896d-1fd19861e72e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.123282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af152522-33c1-409f-896d-1fd19861e72e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.145772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af152522-33c1-409f-896d-1fd19861e72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-af152522-33c1-409f-896d-1fd19861e72e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:55.158788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:55.160662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:55.162210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:55.162210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:55.162210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:55.162210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:55.162210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:55.162210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:55.162210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:55.162210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:55.162210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:55.162210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:55.162210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:55.162210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:55.166941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:55.167875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 197/405] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:55.169617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af152522-33c1-409f-896d-1fd19861e72e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.199757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8785ce2-9dd6-4c48-8c1d-e47cfbcdc6d9 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.242504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee182c-2d0a-4ce6-bde6-4eae5c35f87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.243689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee182c-2d0a-4ce6-bde6-4eae5c35f87c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.244241 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee182c-2d0a-4ce6-bde6-4eae5c35f87c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.244778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee182c-2d0a-4ce6-bde6-4eae5c35f87c None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.248753 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee182c-2d0a-4ce6-bde6-4eae5c35f87c None admin] RBAC: Adding query filter params (name=member) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:55.249353 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee182c-2d0a-4ce6-bde6-4eae5c35f87c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.251911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee182c-2d0a-4ce6-bde6-4eae5c35f87c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.269077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 209/406] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:55.270638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee182c-2d0a-4ce6-bde6-4eae5c35f87c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.396887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af152522-33c1-409f-896d-1fd19861e72e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.398207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee182c-2d0a-4ce6-bde6-4eae5c35f87c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.437197 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dd2ce-d714-4bd9-a66d-28751b034c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.437961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dd2ce-d714-4bd9-a66d-28751b034c53 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.438247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dd2ce-d714-4bd9-a66d-28751b034c53 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.438578 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dd2ce-d714-4bd9-a66d-28751b034c53 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.439386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049dd2ce-d714-4bd9-a66d-28751b034c53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:55.439759 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049dd2ce-d714-4bd9-a66d-28751b034c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.441305 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049dd2ce-d714-4bd9-a66d-28751b034c53 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.455247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:55.456003 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 210/407] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:55.457150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dd2ce-d714-4bd9-a66d-28751b034c53 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.460255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:55.476121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448d166-0689-489e-9928-c57b6b657502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.477651 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448d166-0689-489e-9928-c57b6b657502 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.478457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448d166-0689-489e-9928-c57b6b657502 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.480421 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448d166-0689-489e-9928-c57b6b657502 None admin] PATH_INFO: `/v3/projects/d473eacf42fc47c2aa82e80873c7637e` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.483449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448d166-0689-489e-9928-c57b6b657502 None admin] RBAC: Authorizing `identity:get_project(project_id=d473eacf42fc47c2aa82e80873c7637e)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.486714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448d166-0689-489e-9928-c57b6b657502 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.490624 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 198/408] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/projects/d473eacf42fc47c2aa82e80873c7637e => generated 344 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:55.492494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448d166-0689-489e-9928-c57b6b657502 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.583693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dd2ce-d714-4bd9-a66d-28751b034c53 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.630926 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c993307-f09a-4c61-a866-fd310db77920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.631658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c993307-f09a-4c61-a866-fd310db77920 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.631927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c993307-f09a-4c61-a866-fd310db77920 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.632210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c993307-f09a-4c61-a866-fd310db77920 None admin] PATH_INFO: `/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.633497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c993307-f09a-4c61-a866-fd310db77920 None admin] RBAC: Authorizing `identity:get_group(group_id=9662edef3cdd4b36b22b5a0921c8d3db)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.635010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c993307-f09a-4c61-a866-fd310db77920 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.643406 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 211/409] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:55.643406 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c993307-f09a-4c61-a866-fd310db77920 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.688887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448d166-0689-489e-9928-c57b6b657502 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.730373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be20d62-c5ae-430e-aed4-52a5a529c102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.731021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20d62-c5ae-430e-aed4-52a5a529c102 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.731333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20d62-c5ae-430e-aed4-52a5a529c102 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.731608 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20d62-c5ae-430e-aed4-52a5a529c102 None admin] PATH_INFO: `/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.732954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be20d62-c5ae-430e-aed4-52a5a529c102 None admin] RBAC: Authorizing `identity:get_group(group_id=9662edef3cdd4b36b22b5a0921c8d3db)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.734576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be20d62-c5ae-430e-aed4-52a5a529c102 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.740994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 199/410] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:55.740994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624296d-b34a-4f76-a907-7b07aa0d0e81 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.740994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624296d-b34a-4f76-a907-7b07aa0d0e81 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.740994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624296d-b34a-4f76-a907-7b07aa0d0e81 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.740994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 200/411] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:55.740994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624296d-b34a-4f76-a907-7b07aa0d0e81 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.761935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:55.813442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c993307-f09a-4c61-a866-fd310db77920 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.819722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:55.852087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd84b2-7b78-4ae0-87f5-bf835c3e4d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.852653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd84b2-7b78-4ae0-87f5-bf835c3e4d1f None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.852935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd84b2-7b78-4ae0-87f5-bf835c3e4d1f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.853215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd84b2-7b78-4ae0-87f5-bf835c3e4d1f None admin] PATH_INFO: `/v3/projects/d473eacf42fc47c2aa82e80873c7637e/groups/04ed80dd97d24094b4d269b4599608d1/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.855829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd84b2-7b78-4ae0-87f5-bf835c3e4d1f None admin] RBAC: Authorizing `identity:create_grant(project_id=d473eacf42fc47c2aa82e80873c7637e, group_id=04ed80dd97d24094b4d269b4599608d1, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.857287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd84b2-7b78-4ae0-87f5-bf835c3e4d1f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.865042 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624296d-b34a-4f76-a907-7b07aa0d0e81 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.875017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 212/412] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Aug 31 09:34:55 2022] PUT /identity/v3/projects/d473eacf42fc47c2aa82e80873c7637e/groups/04ed80dd97d24094b4d269b4599608d1/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:55.876357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd84b2-7b78-4ae0-87f5-bf835c3e4d1f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.923974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3054e-8136-444c-a37c-4c623377194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.925336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3054e-8136-444c-a37c-4c623377194e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.925627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3054e-8136-444c-a37c-4c623377194e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.925931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3054e-8136-444c-a37c-4c623377194e None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.927962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3054e-8136-444c-a37c-4c623377194e None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.929481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3054e-8136-444c-a37c-4c623377194e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.937386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 201/413] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:55.937386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502c109-5e9a-4e11-8ee5-b044d8ffd486 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.937386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502c109-5e9a-4e11-8ee5-b044d8ffd486 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.937386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502c109-5e9a-4e11-8ee5-b044d8ffd486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.959454 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-2502c109-5e9a-4e11-8ee5-b044d8ffd486 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:55.960869 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2502c109-5e9a-4e11-8ee5-b044d8ffd486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:56.037316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd84b2-7b78-4ae0-87f5-bf835c3e4d1f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.050959 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 202/414] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:56.054515 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e29a1-23af-46e1-8750-a5045c8576c0 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.055016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e29a1-23af-46e1-8750-a5045c8576c0 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.055558 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e29a1-23af-46e1-8750-a5045c8576c0 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.058845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 203/415] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:56.064336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e29a1-23af-46e1-8750-a5045c8576c0 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.109054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.128013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d4068-e3ee-4c93-a835-7f7299391048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.128797 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d4068-e3ee-4c93-a835-7f7299391048 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.129151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d4068-e3ee-4c93-a835-7f7299391048 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.129504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d4068-e3ee-4c93-a835-7f7299391048 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.130997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d4068-e3ee-4c93-a835-7f7299391048 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:56.132640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d4068-e3ee-4c93-a835-7f7299391048 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.134893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 213/416] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:56.137117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789238e-1a92-4ba5-950e-fea96885b6cc None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.137499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789238e-1a92-4ba5-950e-fea96885b6cc None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.137847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789238e-1a92-4ba5-950e-fea96885b6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.159866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-5789238e-1a92-4ba5-950e-fea96885b6cc None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:56.161221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-5789238e-1a92-4ba5-950e-fea96885b6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:56.178356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.247205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e29a1-23af-46e1-8750-a5045c8576c0 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.287463 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 214/417] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:56.288613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789238e-1a92-4ba5-950e-fea96885b6cc None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.328969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88478f5-487a-42e9-b964-d0896cea9c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.330234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88478f5-487a-42e9-b964-d0896cea9c3a None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.330723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88478f5-487a-42e9-b964-d0896cea9c3a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.331388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88478f5-487a-42e9-b964-d0896cea9c3a None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3/groups/9662edef3cdd4b36b22b5a0921c8d3db/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.335116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88478f5-487a-42e9-b964-d0896cea9c3a None admin] RBAC: Authorizing `identity:create_grant(project_id=7d8956a48042444687dd3ed3e39b1cd3, group_id=9662edef3cdd4b36b22b5a0921c8d3db, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:56.337926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88478f5-487a-42e9-b964-d0896cea9c3a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.366942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 204/418] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Aug 31 09:34:56 2022] PUT /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3/groups/9662edef3cdd4b36b22b5a0921c8d3db/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:56.522817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789238e-1a92-4ba5-950e-fea96885b6cc None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.579508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.583466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88478f5-487a-42e9-b964-d0896cea9c3a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.596164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9e06a-9ac5-4444-adbd-1990c96eb66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.597403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9e06a-9ac5-4444-adbd-1990c96eb66a None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.597863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9e06a-9ac5-4444-adbd-1990c96eb66a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.598324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9e06a-9ac5-4444-adbd-1990c96eb66a None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3/groups/9662edef3cdd4b36b22b5a0921c8d3db/roles/83a1e8de33194004a2a672104465c819` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.602042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9e06a-9ac5-4444-adbd-1990c96eb66a None admin] RBAC: Authorizing `identity:create_grant(project_id=7d8956a48042444687dd3ed3e39b1cd3, group_id=9662edef3cdd4b36b22b5a0921c8d3db, role_id=83a1e8de33194004a2a672104465c819)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:56.604819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9e06a-9ac5-4444-adbd-1990c96eb66a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.634126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 215/419] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Aug 31 09:34:56 2022] PUT /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3/groups/9662edef3cdd4b36b22b5a0921c8d3db/roles/83a1e8de33194004a2a672104465c819 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:56.724151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9e06a-9ac5-4444-adbd-1990c96eb66a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.795474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88478f5-487a-42e9-b964-d0896cea9c3a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.828632 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.845031 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdceaf-ea4a-4cb2-8523-1b047e5ab1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.845886 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdceaf-ea4a-4cb2-8523-1b047e5ab1f0 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.846786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdceaf-ea4a-4cb2-8523-1b047e5ab1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.847219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdceaf-ea4a-4cb2-8523-1b047e5ab1f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.863659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdceaf-ea4a-4cb2-8523-1b047e5ab1f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-04bdceaf-ea4a-4cb2-8523-1b047e5ab1f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:56.869722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:56.873113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:34:56.875054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 205/420] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:56.880272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdceaf-ea4a-4cb2-8523-1b047e5ab1f0 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.958880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9e06a-9ac5-4444-adbd-1990c96eb66a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.997662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdceaf-ea4a-4cb2-8523-1b047e5ab1f0 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.049341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0167a17-fbcd-4004-a715-9a8476adbb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:57.050345 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0167a17-fbcd-4004-a715-9a8476adbb12 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:57.050789 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0167a17-fbcd-4004-a715-9a8476adbb12 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:57.051290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0167a17-fbcd-4004-a715-9a8476adbb12 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:57.052478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0167a17-fbcd-4004-a715-9a8476adbb12 None admin] RBAC: Adding query filter params (name=member) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:57.053003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0167a17-fbcd-4004-a715-9a8476adbb12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:57.055596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0167a17-fbcd-4004-a715-9a8476adbb12 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:57.097572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 206/421] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:57.100677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0167a17-fbcd-4004-a715-9a8476adbb12 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:57.107915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ea8d3-ee0a-465e-beec-0dc5d4cb1c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:57.108705 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ea8d3-ee0a-465e-beec-0dc5d4cb1c4b None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:57.109145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ea8d3-ee0a-465e-beec-0dc5d4cb1c4b None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:57.109591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ea8d3-ee0a-465e-beec-0dc5d4cb1c4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:57.148816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ea8d3-ee0a-465e-beec-0dc5d4cb1c4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-5b6ea8d3-ee0a-465e-beec-0dc5d4cb1c4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:57.159089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:57.160083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:57.160869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:57.160869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:57.160869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:57.160869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:57.160869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:57.160869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:57.160869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:57.160869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:57.160869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:57.160869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:57.160869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:57.160869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:57.176239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 216/422] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 449 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:57.177918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ea8d3-ee0a-465e-beec-0dc5d4cb1c4b None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:57.264461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:57.317392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:57.381768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0167a17-fbcd-4004-a715-9a8476adbb12 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.444024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772eac8-d81a-42b4-a304-26d08f830e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:57.445298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772eac8-d81a-42b4-a304-26d08f830e26 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:57.445945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772eac8-d81a-42b4-a304-26d08f830e26 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:57.446630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772eac8-d81a-42b4-a304-26d08f830e26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:57.511343 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ea8d3-ee0a-465e-beec-0dc5d4cb1c4b None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.578849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772eac8-d81a-42b4-a304-26d08f830e26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:57.581826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772eac8-d81a-42b4-a304-26d08f830e26 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:57.652885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f5623-caa7-42d0-a086-ea9855e57339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:57.653878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f5623-caa7-42d0-a086-ea9855e57339 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:57.654342 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f5623-caa7-42d0-a086-ea9855e57339 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:57.654797 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f5623-caa7-42d0-a086-ea9855e57339 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:57.655964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f5623-caa7-42d0-a086-ea9855e57339 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:57.656702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f5623-caa7-42d0-a086-ea9855e57339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:57.659630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f5623-caa7-42d0-a086-ea9855e57339 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:57.671529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:57.702884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 217/423] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 526 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:57.708513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f5623-caa7-42d0-a086-ea9855e57339 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:57.770994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 207/424] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:57.778031 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772eac8-d81a-42b4-a304-26d08f830e26 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:57.970638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f5623-caa7-42d0-a086-ea9855e57339 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.989208 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772eac8-d81a-42b4-a304-26d08f830e26 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.992374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:58.050140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4d737-0ee9-4bb7-a1e5-3288972645be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.050570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4d737-0ee9-4bb7-a1e5-3288972645be None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.050700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4d737-0ee9-4bb7-a1e5-3288972645be None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.051047 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4d737-0ee9-4bb7-a1e5-3288972645be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.067017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c388d-8f82-4509-ab76-55394a71c045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.068677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c388d-8f82-4509-ab76-55394a71c045 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.069521 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c388d-8f82-4509-ab76-55394a71c045 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.070258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c388d-8f82-4509-ab76-55394a71c045 None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.073013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c388d-8f82-4509-ab76-55394a71c045 None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.077249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c388d-8f82-4509-ab76-55394a71c045 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.081792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 208/425] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:57 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:58.100844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c388d-8f82-4509-ab76-55394a71c045 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:58.155055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:58.262814 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4d737-0ee9-4bb7-a1e5-3288972645be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.264366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4d737-0ee9-4bb7-a1e5-3288972645be None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.365514 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c388d-8f82-4509-ab76-55394a71c045 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:58.438479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28710a-3b86-4998-8142-829d898219d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.439575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28710a-3b86-4998-8142-829d898219d2 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.440060 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28710a-3b86-4998-8142-829d898219d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.440553 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28710a-3b86-4998-8142-829d898219d2 None admin] PATH_INFO: `/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.444390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 218/426] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 737 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:58.445694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28710a-3b86-4998-8142-829d898219d2 None admin] RBAC: Authorizing `identity:get_group(group_id=9662edef3cdd4b36b22b5a0921c8d3db)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.447452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28710a-3b86-4998-8142-829d898219d2 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.451171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4d737-0ee9-4bb7-a1e5-3288972645be None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:58.454013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 209/427] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:58 2022] GET /identity/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db => generated 226 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:58.465810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28710a-3b86-4998-8142-829d898219d2 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:58.468489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:58.568233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:58.602565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28710a-3b86-4998-8142-829d898219d2 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:58.643028 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4d737-0ee9-4bb7-a1e5-3288972645be None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:58.665003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eeefdc-97ef-40f8-9692-d9cdff2cae31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.665624 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeefdc-97ef-40f8-9692-d9cdff2cae31 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.665901 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeefdc-97ef-40f8-9692-d9cdff2cae31 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.666172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeefdc-97ef-40f8-9692-d9cdff2cae31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.666963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeefdc-97ef-40f8-9692-d9cdff2cae31 None admin] RBAC: Adding query filter params (group.id=9662edef3cdd4b36b22b5a0921c8d3db, role.id=7ccf7639a4864595b7bc589a9dc44df7, scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:58.667309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeefdc-97ef-40f8-9692-d9cdff2cae31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.668812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeefdc-97ef-40f8-9692-d9cdff2cae31 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.676538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 210/428] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 09:34:58 2022] GET /identity/v3/role_assignments?group.id=9662edef3cdd4b36b22b5a0921c8d3db&scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2&role.id=7ccf7639a4864595b7bc589a9dc44df7 => generated 611 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:58.677950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c0f63-94e2-43f5-9bc3-83c0fec76760 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.678242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c0f63-94e2-43f5-9bc3-83c0fec76760 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.678516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c0f63-94e2-43f5-9bc3-83c0fec76760 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.679996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 211/429] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:58.686648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bbce3-9d74-4d50-a9be-17a9f44697d6 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.686956 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bbce3-9d74-4d50-a9be-17a9f44697d6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.687238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bbce3-9d74-4d50-a9be-17a9f44697d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.729680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-511bbce3-9d74-4d50-a9be-17a9f44697d6 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:58.732118 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-511bbce3-9d74-4d50-a9be-17a9f44697d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:58.737449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e8a75-ffb1-4968-8b7b-80b3442c4b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.738585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e8a75-ffb1-4968-8b7b-80b3442c4b1a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.738980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e8a75-ffb1-4968-8b7b-80b3442c4b1a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.739460 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e8a75-ffb1-4968-8b7b-80b3442c4b1a None admin] PATH_INFO: `/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.741485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e8a75-ffb1-4968-8b7b-80b3442c4b1a None admin] RBAC: Authorizing `identity:get_project(project_id=76ea9635caef4c68bc934ba9f9ee98f2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.744305 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e8a75-ffb1-4968-8b7b-80b3442c4b1a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.748656 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 219/430] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:58 2022] GET /identity/v3/projects/76ea9635caef4c68bc934ba9f9ee98f2 => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:58.761473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e8a75-ffb1-4968-8b7b-80b3442c4b1a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:58.900783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:58.903633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 212/431] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:58.926392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e8a75-ffb1-4968-8b7b-80b3442c4b1a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:58.966246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ed21f-d2f8-4add-83fe-7c942a5d2089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.966904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ed21f-d2f8-4add-83fe-7c942a5d2089 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.967211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ed21f-d2f8-4add-83fe-7c942a5d2089 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.967493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ed21f-d2f8-4add-83fe-7c942a5d2089 None admin] PATH_INFO: `/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.968730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ed21f-d2f8-4add-83fe-7c942a5d2089 None admin] RBAC: Authorizing `identity:get_group(group_id=9662edef3cdd4b36b22b5a0921c8d3db)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.970200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ed21f-d2f8-4add-83fe-7c942a5d2089 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.972209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 220/432] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:58 2022] GET /identity/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:58.977767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bbce3-9d74-4d50-a9be-17a9f44697d6 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:59.036839 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e017be-0939-4f03-a3b9-7e65b4b0efdd None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.037167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e017be-0939-4f03-a3b9-7e65b4b0efdd None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.037489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e017be-0939-4f03-a3b9-7e65b4b0efdd None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.041756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 221/433] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:59.050713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6576-7b94-4515-a5bd-8b21f7ec5710 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.050997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6576-7b94-4515-a5bd-8b21f7ec5710 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.051476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6576-7b94-4515-a5bd-8b21f7ec5710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.074704 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ac5f6576-7b94-4515-a5bd-8b21f7ec5710 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:59.075278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ac5f6576-7b94-4515-a5bd-8b21f7ec5710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:59.088732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:59.153262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bbce3-9d74-4d50-a9be-17a9f44697d6 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:59.215375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 222/434] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:59.223204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f6576-7b94-4515-a5bd-8b21f7ec5710 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:59.230885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f09d79-eac8-459d-8ea8-39a1cae60fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:59.231802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f09d79-eac8-459d-8ea8-39a1cae60fef None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.232260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f09d79-eac8-459d-8ea8-39a1cae60fef None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.233000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f09d79-eac8-459d-8ea8-39a1cae60fef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.234127 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f09d79-eac8-459d-8ea8-39a1cae60fef None admin] RBAC: Adding query filter params (group.id=9662edef3cdd4b36b22b5a0921c8d3db, role.id=83a1e8de33194004a2a672104465c819, scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:59.234652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f09d79-eac8-459d-8ea8-39a1cae60fef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:59.236598 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f09d79-eac8-459d-8ea8-39a1cae60fef None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:59.245680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 213/435] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 09:34:58 2022] GET /identity/v3/role_assignments?group.id=9662edef3cdd4b36b22b5a0921c8d3db&scope.project.id=76ea9635caef4c68bc934ba9f9ee98f2&role.id=83a1e8de33194004a2a672104465c819 => generated 611 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:59.342503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:59.375337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f6576-7b94-4515-a5bd-8b21f7ec5710 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:59.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a112e-a238-4210-a6ad-5ab60100f06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:59.424741 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a112e-a238-4210-a6ad-5ab60100f06a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.425019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a112e-a238-4210-a6ad-5ab60100f06a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.425292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a112e-a238-4210-a6ad-5ab60100f06a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.439065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a112e-a238-4210-a6ad-5ab60100f06a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-c81a112e-a238-4210-a6ad-5ab60100f06a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:59.444123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:59.445074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:59.446328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 223/436] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:59.454296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f09d79-eac8-459d-8ea8-39a1cae60fef None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:59.623046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f09d79-eac8-459d-8ea8-39a1cae60fef None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:59.644497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a112e-a238-4210-a6ad-5ab60100f06a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:59.686322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77a86a-0dbc-44c0-94cb-948882bf589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:59.688562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77a86a-0dbc-44c0-94cb-948882bf589b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.690176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77a86a-0dbc-44c0-94cb-948882bf589b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.691106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77a86a-0dbc-44c0-94cb-948882bf589b None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.692722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77a86a-0dbc-44c0-94cb-948882bf589b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:59.693681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77a86a-0dbc-44c0-94cb-948882bf589b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:59.696805 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77a86a-0dbc-44c0-94cb-948882bf589b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:59.710094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:59.724819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 214/437] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:59.724819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d681d-97b1-4dd8-936e-b5a00e492656 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.724819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d681d-97b1-4dd8-936e-b5a00e492656 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.724819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d681d-97b1-4dd8-936e-b5a00e492656 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.724819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 215/438] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:59.730453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d94a8e-4f13-486b-9778-d6d244188af9 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.730991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d94a8e-4f13-486b-9778-d6d244188af9 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.731466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d94a8e-4f13-486b-9778-d6d244188af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.761600 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-97d94a8e-4f13-486b-9778-d6d244188af9 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:59.764347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-97d94a8e-4f13-486b-9778-d6d244188af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:59.786249 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a112e-a238-4210-a6ad-5ab60100f06a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:59.830031 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fad028-a84a-489f-81e9-83702aec23d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:59.830418 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fad028-a84a-489f-81e9-83702aec23d7 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.830696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fad028-a84a-489f-81e9-83702aec23d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.830970 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fad028-a84a-489f-81e9-83702aec23d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.835173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:59.847290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fad028-a84a-489f-81e9-83702aec23d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-90fad028-a84a-489f-81e9-83702aec23d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:59.852589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:59.853620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:59.857856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:59.857856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:59.857856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:59.857856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:59.857856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:59.857856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:59.857856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:59.857856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:59.857856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:59.857856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:59.857856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:59.857856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:34:59.860649 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 224/439] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:59.861187 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fad028-a84a-489f-81e9-83702aec23d7 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:59.911546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 216/440] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:59.912842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d94a8e-4f13-486b-9778-d6d244188af9 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.064208 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d94a8e-4f13-486b-9778-d6d244188af9 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.087497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fad028-a84a-489f-81e9-83702aec23d7 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.105827 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:00.118175 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281a84a-7050-43be-b62a-7d3a7cb07911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.119273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281a84a-7050-43be-b62a-7d3a7cb07911 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.119767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281a84a-7050-43be-b62a-7d3a7cb07911 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.121861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281a84a-7050-43be-b62a-7d3a7cb07911 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.123612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281a84a-7050-43be-b62a-7d3a7cb07911 None admin] RBAC: Adding query filter params (name=member) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:00.124162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281a84a-7050-43be-b62a-7d3a7cb07911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.126770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281a84a-7050-43be-b62a-7d3a7cb07911 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.148183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 217/441] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:00.152897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7076d01-077a-4832-9893-6cd34909c65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.153551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7076d01-077a-4832-9893-6cd34909c65e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.153803 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7076d01-077a-4832-9893-6cd34909c65e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.154073 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7076d01-077a-4832-9893-6cd34909c65e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.159282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281a84a-7050-43be-b62a-7d3a7cb07911 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.268267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:00.280198 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7076d01-077a-4832-9893-6cd34909c65e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.282954 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7076d01-077a-4832-9893-6cd34909c65e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.284065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281a84a-7050-43be-b62a-7d3a7cb07911 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.331747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d36fa-4e25-400e-a5e7-26c17e4cf175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.332387 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d36fa-4e25-400e-a5e7-26c17e4cf175 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.332693 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d36fa-4e25-400e-a5e7-26c17e4cf175 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.333010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d36fa-4e25-400e-a5e7-26c17e4cf175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.416480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 225/442] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:00.417924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7076d01-077a-4832-9893-6cd34909c65e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.424475 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:00.439164 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d36fa-4e25-400e-a5e7-26c17e4cf175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.440866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d36fa-4e25-400e-a5e7-26c17e4cf175 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.560228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:00.570007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7076d01-077a-4832-9893-6cd34909c65e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.572908 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 218/443] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:35:00.573832 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d36fa-4e25-400e-a5e7-26c17e4cf175 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.628629 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8ccf6-36b2-49cc-ab98-34bbe77049a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.629567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8ccf6-36b2-49cc-ab98-34bbe77049a6 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.630038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8ccf6-36b2-49cc-ab98-34bbe77049a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.630378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8ccf6-36b2-49cc-ab98-34bbe77049a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.642319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8ccf6-36b2-49cc-ab98-34bbe77049a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-f1e8ccf6-36b2-49cc-ab98-34bbe77049a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:00.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:00.648975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 226/444] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:00.650724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8ccf6-36b2-49cc-ab98-34bbe77049a6 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.681804 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:00.696608 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d36fa-4e25-400e-a5e7-26c17e4cf175 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.747811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5545e-4d6f-4510-8f9e-6077f5cc0c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.748466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5545e-4d6f-4510-8f9e-6077f5cc0c7e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.748738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5545e-4d6f-4510-8f9e-6077f5cc0c7e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.749004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5545e-4d6f-4510-8f9e-6077f5cc0c7e None admin] PATH_INFO: `/v3/projects/d473eacf42fc47c2aa82e80873c7637e` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.750648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5545e-4d6f-4510-8f9e-6077f5cc0c7e None admin] RBAC: Authorizing `identity:get_project(project_id=d473eacf42fc47c2aa82e80873c7637e)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.752488 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5545e-4d6f-4510-8f9e-6077f5cc0c7e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.754174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 219/445] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/projects/d473eacf42fc47c2aa82e80873c7637e => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:00.754952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5545e-4d6f-4510-8f9e-6077f5cc0c7e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.790167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8ccf6-36b2-49cc-ab98-34bbe77049a6 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.830207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9f65c-e199-4054-8172-a9ad9c156849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.830976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9f65c-e199-4054-8172-a9ad9c156849 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.831346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9f65c-e199-4054-8172-a9ad9c156849 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.831681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9f65c-e199-4054-8172-a9ad9c156849 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.833005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9f65c-e199-4054-8172-a9ad9c156849 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.834621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9f65c-e199-4054-8172-a9ad9c156849 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.836412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 227/446] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:00.837348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9f65c-e199-4054-8172-a9ad9c156849 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.867437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:00.892866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5545e-4d6f-4510-8f9e-6077f5cc0c7e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.928183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7230146-8470-4c65-8eec-8c41a302bd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.928793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7230146-8470-4c65-8eec-8c41a302bd4f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.929067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7230146-8470-4c65-8eec-8c41a302bd4f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.929333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7230146-8470-4c65-8eec-8c41a302bd4f None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.929996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7230146-8470-4c65-8eec-8c41a302bd4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:00.930292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7230146-8470-4c65-8eec-8c41a302bd4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.931711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7230146-8470-4c65-8eec-8c41a302bd4f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.940813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 220/447] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:00.941649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7230146-8470-4c65-8eec-8c41a302bd4f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.960343 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9f65c-e199-4054-8172-a9ad9c156849 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.976875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:01.004808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbf17b-d488-489d-851c-5323b7e175b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.005442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf17b-d488-489d-851c-5323b7e175b3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.005725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf17b-d488-489d-851c-5323b7e175b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.006000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf17b-d488-489d-851c-5323b7e175b3 None admin] PATH_INFO: `/v3/groups/04ed80dd97d24094b4d269b4599608d1` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.007176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbf17b-d488-489d-851c-5323b7e175b3 None admin] RBAC: Authorizing `identity:get_group(group_id=04ed80dd97d24094b4d269b4599608d1)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.008619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbf17b-d488-489d-851c-5323b7e175b3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.010323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 228/448] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/groups/04ed80dd97d24094b4d269b4599608d1 => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:01.011932 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbf17b-d488-489d-851c-5323b7e175b3 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.069509 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7230146-8470-4c65-8eec-8c41a302bd4f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.113359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561175-3dc1-4f68-b245-3b3f00eb499b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.114063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561175-3dc1-4f68-b245-3b3f00eb499b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.114400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561175-3dc1-4f68-b245-3b3f00eb499b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.114730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561175-3dc1-4f68-b245-3b3f00eb499b None admin] PATH_INFO: `/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.116288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e561175-3dc1-4f68-b245-3b3f00eb499b None admin] RBAC: Authorizing `identity:get_group(group_id=9662edef3cdd4b36b22b5a0921c8d3db)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.117962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e561175-3dc1-4f68-b245-3b3f00eb499b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.128013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 221/449] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:01.128013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561175-3dc1-4f68-b245-3b3f00eb499b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.139651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:01.152795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbf17b-d488-489d-851c-5323b7e175b3 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.190940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75186c7c-c453-4d63-8681-8c4774445043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.191586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75186c7c-c453-4d63-8681-8c4774445043 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.191858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75186c7c-c453-4d63-8681-8c4774445043 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.192124 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75186c7c-c453-4d63-8681-8c4774445043 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.257274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:01.271275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561175-3dc1-4f68-b245-3b3f00eb499b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.277258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75186c7c-c453-4d63-8681-8c4774445043 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.280103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75186c7c-c453-4d63-8681-8c4774445043 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.312400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358eecc-f4b6-4993-ba8c-85fd2f27d2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.313142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358eecc-f4b6-4993-ba8c-85fd2f27d2d1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.313478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358eecc-f4b6-4993-ba8c-85fd2f27d2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.313856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358eecc-f4b6-4993-ba8c-85fd2f27d2d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.314704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358eecc-f4b6-4993-ba8c-85fd2f27d2d1 None admin] RBAC: Adding query filter params (group.id=04ed80dd97d24094b4d269b4599608d1, role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=d473eacf42fc47c2aa82e80873c7637e) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:01.315160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358eecc-f4b6-4993-ba8c-85fd2f27d2d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.316734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358eecc-f4b6-4993-ba8c-85fd2f27d2d1 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.324858 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 222/450] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 09:35:01 2022] GET /identity/v3/role_assignments?group.id=04ed80dd97d24094b4d269b4599608d1&scope.project.id=d473eacf42fc47c2aa82e80873c7637e&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:01.325746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358eecc-f4b6-4993-ba8c-85fd2f27d2d1 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.421088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:01.436816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 229/451] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:01.443111 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75186c7c-c453-4d63-8681-8c4774445043 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.474161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358eecc-f4b6-4993-ba8c-85fd2f27d2d1 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.523957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffb058-932e-4816-a925-14490517abb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.525061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb058-932e-4816-a925-14490517abb6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.525531 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb058-932e-4816-a925-14490517abb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.526857 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb058-932e-4816-a925-14490517abb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.528086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffb058-932e-4816-a925-14490517abb6 None admin] RBAC: Adding query filter params (group.id=9662edef3cdd4b36b22b5a0921c8d3db, role.id=7ccf7639a4864595b7bc589a9dc44df7, scope.project.id=7d8956a48042444687dd3ed3e39b1cd3) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:01.528792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffb058-932e-4816-a925-14490517abb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.530747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffb058-932e-4816-a925-14490517abb6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.539529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 223/452] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 09:35:01 2022] GET /identity/v3/role_assignments?group.id=9662edef3cdd4b36b22b5a0921c8d3db&scope.project.id=7d8956a48042444687dd3ed3e39b1cd3&role.id=7ccf7639a4864595b7bc589a9dc44df7 => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:01.598107 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75186c7c-c453-4d63-8681-8c4774445043 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.665882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437dc718-cc16-4e63-b998-32196b44efc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.667635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dc718-cc16-4e63-b998-32196b44efc2 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.668523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dc718-cc16-4e63-b998-32196b44efc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.668523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dc718-cc16-4e63-b998-32196b44efc2 None admin] PATH_INFO: `/v3/projects/7d8956a48042444687dd3ed3e39b1cd3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.670576 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437dc718-cc16-4e63-b998-32196b44efc2 None admin] RBAC: Authorizing `identity:get_project(project_id=7d8956a48042444687dd3ed3e39b1cd3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.673200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437dc718-cc16-4e63-b998-32196b44efc2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.676421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 230/453] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:01 2022] GET /identity/v3/projects/7d8956a48042444687dd3ed3e39b1cd3 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:01.683160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffb058-932e-4816-a925-14490517abb6 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.711041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:01.826807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffb058-932e-4816-a925-14490517abb6 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.886479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7d4ca-c59a-4d68-9eb6-55ffcecd1634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.887810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7d4ca-c59a-4d68-9eb6-55ffcecd1634 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.888311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7d4ca-c59a-4d68-9eb6-55ffcecd1634 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.888808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7d4ca-c59a-4d68-9eb6-55ffcecd1634 None admin] PATH_INFO: `/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.891219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7d4ca-c59a-4d68-9eb6-55ffcecd1634 None admin] RBAC: Authorizing `identity:get_group(group_id=9662edef3cdd4b36b22b5a0921c8d3db)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.893716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7d4ca-c59a-4d68-9eb6-55ffcecd1634 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.897203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 224/454] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 09:35:01 2022] GET /identity/v3/groups/9662edef3cdd4b36b22b5a0921c8d3db => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:01.903422 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437dc718-cc16-4e63-b998-32196b44efc2 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.991621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:02.040070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437dc718-cc16-4e63-b998-32196b44efc2 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:02.081057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3f8aa-ef64-4695-9a9f-7ea23cb15feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:02.081710 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3f8aa-ef64-4695-9a9f-7ea23cb15feb None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:02.082021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3f8aa-ef64-4695-9a9f-7ea23cb15feb None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:02.082327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3f8aa-ef64-4695-9a9f-7ea23cb15feb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:02.083099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3f8aa-ef64-4695-9a9f-7ea23cb15feb None admin] RBAC: Adding query filter params (group.id=9662edef3cdd4b36b22b5a0921c8d3db, role.id=83a1e8de33194004a2a672104465c819, scope.project.id=7d8956a48042444687dd3ed3e39b1cd3) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:02.083487 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3f8aa-ef64-4695-9a9f-7ea23cb15feb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:02.084968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3f8aa-ef64-4695-9a9f-7ea23cb15feb None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:02.092365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 231/455] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 09:35:01 2022] GET /identity/v3/role_assignments?group.id=9662edef3cdd4b36b22b5a0921c8d3db&scope.project.id=7d8956a48042444687dd3ed3e39b1cd3&role.id=83a1e8de33194004a2a672104465c819 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:04.928779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d1700-246e-41d6-bdff-0253effb50a3 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.930052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d1700-246e-41d6-bdff-0253effb50a3 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.930749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d1700-246e-41d6-bdff-0253effb50a3 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.934401 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 225/456] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:04.942092 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54e85f-f696-44cb-b141-c9f59438590e None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.943274 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54e85f-f696-44cb-b141-c9f59438590e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.943866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54e85f-f696-44cb-b141-c9f59438590e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.977832 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0c54e85f-f696-44cb-b141-c9f59438590e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:04.979881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0c54e85f-f696-44cb-b141-c9f59438590e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:05.080951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 232/457] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:05.235501 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9e52d-8597-48c5-935d-27c7eddf73a7 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.236430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9e52d-8597-48c5-935d-27c7eddf73a7 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.237158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9e52d-8597-48c5-935d-27c7eddf73a7 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.240422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 226/458] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:05.246169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c9c2-b042-4276-b2b1-ef0765b90b6e None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.249479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c9c2-b042-4276-b2b1-ef0765b90b6e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.249479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c9c2-b042-4276-b2b1-ef0765b90b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.276285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e674c9c2-b042-4276-b2b1-ef0765b90b6e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:05.277927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e674c9c2-b042-4276-b2b1-ef0765b90b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:05.353609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:05.382091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 233/459] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:05.535106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a014169-bdad-42f8-9f0a-42144a398673 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.535303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a014169-bdad-42f8-9f0a-42144a398673 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.535666 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a014169-bdad-42f8-9f0a-42144a398673 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.537412 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 227/460] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:05.544500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0059bd-8096-4bc2-b5a1-3ca971f2ac55 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.545005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0059bd-8096-4bc2-b5a1-3ca971f2ac55 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.545502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0059bd-8096-4bc2-b5a1-3ca971f2ac55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.584937 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-cf0059bd-8096-4bc2-b5a1-3ca971f2ac55 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:05.586825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-cf0059bd-8096-4bc2-b5a1-3ca971f2ac55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:05.734231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a014169-bdad-42f8-9f0a-42144a398673 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:05.753631 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 234/461] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:05.790861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0059bd-8096-4bc2-b5a1-3ca971f2ac55 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:05.928885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:05.967334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a014169-bdad-42f8-9f0a-42144a398673 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:06.043533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0059bd-8096-4bc2-b5a1-3ca971f2ac55 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:06.044103 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87167a0-e13f-434e-940f-f5ee954831b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:06.044500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87167a0-e13f-434e-940f-f5ee954831b9 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.044764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87167a0-e13f-434e-940f-f5ee954831b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.044998 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87167a0-e13f-434e-940f-f5ee954831b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.060024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87167a0-e13f-434e-940f-f5ee954831b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:06.061511 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87167a0-e13f-434e-940f-f5ee954831b9 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:06.077598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-e87167a0-e13f-434e-940f-f5ee954831b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:06.081006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:06.081899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:06.088094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:06.088094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:06.088094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:06.088094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 228/462] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:06.088094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d02f4c-13e0-4b61-9e48-bb559f0ac7f4 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.088094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d02f4c-13e0-4b61-9e48-bb559f0ac7f4 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.088094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d02f4c-13e0-4b61-9e48-bb559f0ac7f4 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.088094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 229/463] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:06.088094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d02f4c-13e0-4b61-9e48-bb559f0ac7f4 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:06.112667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafc1f4-daf7-4dbd-a6cf-95a689d2256e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:06.113319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafc1f4-daf7-4dbd-a6cf-95a689d2256e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.113599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafc1f4-daf7-4dbd-a6cf-95a689d2256e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.113861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafc1f4-daf7-4dbd-a6cf-95a689d2256e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.132734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafc1f4-daf7-4dbd-a6cf-95a689d2256e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:06.135738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafc1f4-daf7-4dbd-a6cf-95a689d2256e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-3bafc1f4-daf7-4dbd-a6cf-95a689d2256e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:06.158614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:06.159609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:06.160431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:06.160431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:06.160431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:06.163334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 235/464] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:06.164717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafc1f4-daf7-4dbd-a6cf-95a689d2256e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:06.237935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d02f4c-13e0-4b61-9e48-bb559f0ac7f4 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:06.295206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51e0ae-e914-4a30-adc5-5276c78d368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:06.296214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51e0ae-e914-4a30-adc5-5276c78d368c None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.296726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51e0ae-e914-4a30-adc5-5276c78d368c None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.297227 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51e0ae-e914-4a30-adc5-5276c78d368c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.297811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafc1f4-daf7-4dbd-a6cf-95a689d2256e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:06.312016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:06.319213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51e0ae-e914-4a30-adc5-5276c78d368c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:06.321960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51e0ae-e914-4a30-adc5-5276c78d368c None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:06.337491 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67463c85-4741-4323-99cd-db2b114c24c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:06.338501 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67463c85-4741-4323-99cd-db2b114c24c2 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.339647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67463c85-4741-4323-99cd-db2b114c24c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.340149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67463c85-4741-4323-99cd-db2b114c24c2 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.340971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67463c85-4741-4323-99cd-db2b114c24c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:06.341428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67463c85-4741-4323-99cd-db2b114c24c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:06.342958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67463c85-4741-4323-99cd-db2b114c24c2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-cd51e0ae-e914-4a30-adc5-5276c78d368c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:06.350672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:06.351806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:06.352855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:06.352855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:06.352855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:06.352855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 230/465] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:06.355005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27a91c-b437-4ad2-8db2-25323c5b3a5b None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.355537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27a91c-b437-4ad2-8db2-25323c5b3a5b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.356021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27a91c-b437-4ad2-8db2-25323c5b3a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.365358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 236/466] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:06.366389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67463c85-4741-4323-99cd-db2b114c24c2 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:06.387824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4a27a91c-b437-4ad2-8db2-25323c5b3a5b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:06.389450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4a27a91c-b437-4ad2-8db2-25323c5b3a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:06.405691 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:06.478947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 231/467] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:06.480418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27a91c-b437-4ad2-8db2-25323c5b3a5b None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:06.526183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67463c85-4741-4323-99cd-db2b114c24c2 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:06.578295 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f571a-8699-46c6-aa41-38b786e7d4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:06.578889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f571a-8699-46c6-aa41-38b786e7d4ef None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.579205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f571a-8699-46c6-aa41-38b786e7d4ef None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.579516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f571a-8699-46c6-aa41-38b786e7d4ef None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.580941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f571a-8699-46c6-aa41-38b786e7d4ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:06.581248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f571a-8699-46c6-aa41-38b786e7d4ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:06.583460 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f571a-8699-46c6-aa41-38b786e7d4ef None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:06.599690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 237/468] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:06.600977 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f571a-8699-46c6-aa41-38b786e7d4ef None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:06.613044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27a91c-b437-4ad2-8db2-25323c5b3a5b None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:06.639630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:06.657244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22b2ab-8f2f-4702-aa7a-71edab5a6fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:06.657868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22b2ab-8f2f-4702-aa7a-71edab5a6fc7 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.658144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22b2ab-8f2f-4702-aa7a-71edab5a6fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.658415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22b2ab-8f2f-4702-aa7a-71edab5a6fc7 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.659222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22b2ab-8f2f-4702-aa7a-71edab5a6fc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:06.659534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22b2ab-8f2f-4702-aa7a-71edab5a6fc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:06.660975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22b2ab-8f2f-4702-aa7a-71edab5a6fc7 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:06.682927 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 232/469] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:06.683982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22b2ab-8f2f-4702-aa7a-71edab5a6fc7 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:06.711605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:06.730987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f571a-8699-46c6-aa41-38b786e7d4ef None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:06.773202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362dd9d-bb3e-4d42-ab64-a51da0c0e379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:06.774088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362dd9d-bb3e-4d42-ab64-a51da0c0e379 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.774596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362dd9d-bb3e-4d42-ab64-a51da0c0e379 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.775097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362dd9d-bb3e-4d42-ab64-a51da0c0e379 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.776629 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362dd9d-bb3e-4d42-ab64-a51da0c0e379 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:06.779015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362dd9d-bb3e-4d42-ab64-a51da0c0e379 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:06.803890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22b2ab-8f2f-4702-aa7a-71edab5a6fc7 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:06.814429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 238/470] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:35:06 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:06.824532 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362dd9d-bb3e-4d42-ab64-a51da0c0e379 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:06.936097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08bb55-ba2c-4d9a-adc8-42fca304ca01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:06.936994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08bb55-ba2c-4d9a-adc8-42fca304ca01 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.937402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08bb55-ba2c-4d9a-adc8-42fca304ca01 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.937804 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08bb55-ba2c-4d9a-adc8-42fca304ca01 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.938837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08bb55-ba2c-4d9a-adc8-42fca304ca01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:06.950069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08bb55-ba2c-4d9a-adc8-42fca304ca01 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:06.977751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 233/471] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:35:06 2022] POST /identity/v3/users => generated 246 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:06.979204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08bb55-ba2c-4d9a-adc8-42fca304ca01 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:07.033620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:07.138742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08bb55-ba2c-4d9a-adc8-42fca304ca01 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:07.146319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362dd9d-bb3e-4d42-ab64-a51da0c0e379 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:07.202189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3eca38-c622-4a86-832f-ed601e33e41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:07.202914 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3eca38-c622-4a86-832f-ed601e33e41c None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.203342 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3eca38-c622-4a86-832f-ed601e33e41c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.203730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3eca38-c622-4a86-832f-ed601e33e41c None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.204518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3eca38-c622-4a86-832f-ed601e33e41c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:07.207064 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3eca38-c622-4a86-832f-ed601e33e41c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:07.217304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084d668-9ff1-48a9-853b-6e0952e631aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:07.219422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084d668-9ff1-48a9-853b-6e0952e631aa None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.220454 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084d668-9ff1-48a9-853b-6e0952e631aa None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.220874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084d668-9ff1-48a9-853b-6e0952e631aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.233359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:07.242562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 239/472] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:35:06 2022] POST /identity/v3/users => generated 249 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:07.251156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e8b9-9d0e-48bf-b06a-2c366caeb592 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.251156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e8b9-9d0e-48bf-b06a-2c366caeb592 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.251156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e8b9-9d0e-48bf-b06a-2c366caeb592 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.251635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084d668-9ff1-48a9-853b-6e0952e631aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:07.251751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 240/473] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:07.251861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084d668-9ff1-48a9-853b-6e0952e631aa None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:07.259110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248d068-0308-4bbd-af1f-e5f1ac07b882 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.259772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248d068-0308-4bbd-af1f-e5f1ac07b882 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.260070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248d068-0308-4bbd-af1f-e5f1ac07b882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.285091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b248d068-0308-4bbd-af1f-e5f1ac07b882 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:07.286506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b248d068-0308-4bbd-af1f-e5f1ac07b882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-2084d668-9ff1-48a9-853b-6e0952e631aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:07.290081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:07.292485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:07.294835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:07.294835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:07.294835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:07.298025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 234/474] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:07.310214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084d668-9ff1-48a9-853b-6e0952e631aa None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:07.421119 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 241/475] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:07.497928 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:07.513073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084d668-9ff1-48a9-853b-6e0952e631aa None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:07.558587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e57b15-c015-467c-aaf3-f95de52bd2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:07.559288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e57b15-c015-467c-aaf3-f95de52bd2c7 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.559582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e57b15-c015-467c-aaf3-f95de52bd2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.559849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e57b15-c015-467c-aaf3-f95de52bd2c7 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.560566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e57b15-c015-467c-aaf3-f95de52bd2c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:07.560875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e57b15-c015-467c-aaf3-f95de52bd2c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:07.562299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e57b15-c015-467c-aaf3-f95de52bd2c7 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:07.578017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 235/476] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:07.583816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248d068-0308-4bbd-af1f-e5f1ac07b882 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:07.716628 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e57b15-c015-467c-aaf3-f95de52bd2c7 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:07.797979 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248d068-0308-4bbd-af1f-e5f1ac07b882 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:07.864987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:07.883801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52d44e-4e9a-4615-94d1-1cfeaf52e527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:07.884418 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52d44e-4e9a-4615-94d1-1cfeaf52e527 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.884694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52d44e-4e9a-4615-94d1-1cfeaf52e527 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.887451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52d44e-4e9a-4615-94d1-1cfeaf52e527 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.887451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52d44e-4e9a-4615-94d1-1cfeaf52e527 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:07.888693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52d44e-4e9a-4615-94d1-1cfeaf52e527 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:07.918558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 242/477] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:35:07 2022] POST /identity/v3/users => generated 248 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:07.939841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e57b15-c015-467c-aaf3-f95de52bd2c7 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:08.001922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cc7ef-da47-464a-ab50-5e8e2381e5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:08.003029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cc7ef-da47-464a-ab50-5e8e2381e5e0 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.003574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cc7ef-da47-464a-ab50-5e8e2381e5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.004037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cc7ef-da47-464a-ab50-5e8e2381e5e0 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.005296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cc7ef-da47-464a-ab50-5e8e2381e5e0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:08.007861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cc7ef-da47-464a-ab50-5e8e2381e5e0 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:08.027733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.sql.core [None req-2d1cc7ef-da47-464a-ab50-5e8e2381e5e0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 09:35:08.027733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 09:35:08.027733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [parameters: {'id': 'e67f8cddbfc9485d8ff29b9d5d284c09', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 09:35:08.027733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93533) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-2d1cc7ef-da47-464a-ab50-5e8e2381e5e0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application result.read() Aug 31 09:35:08.058661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:35:08.060321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application yield resource Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise self.value Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application session.commit() Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 09:35:08.063753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self.session.flush() Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application compat.raise_( Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 09:35:08.068736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:35:08.072216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application result.read() Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:35:08.080260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [parameters: {'id': 'e67f8cddbfc9485d8ff29b9d5d284c09', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:08.084246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 09:35:08.087367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:08.100191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 236/478] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Aug 31 09:35:07 2022] POST /identity/v3/roles => generated 144 bytes in 345 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:08.100488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52d44e-4e9a-4615-94d1-1cfeaf52e527 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:08.243957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52d44e-4e9a-4615-94d1-1cfeaf52e527 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:08.282627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc7ed7-c423-4068-9ebb-ba2f3f7e01c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:08.283060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc7ed7-c423-4068-9ebb-ba2f3f7e01c3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.283060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc7ed7-c423-4068-9ebb-ba2f3f7e01c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.283559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc7ed7-c423-4068-9ebb-ba2f3f7e01c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.294711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc7ed7-c423-4068-9ebb-ba2f3f7e01c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-b6bc7ed7-c423-4068-9ebb-ba2f3f7e01c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:08.300566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:08.304269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:08.312293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 243/479] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:08.314609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cc7ef-da47-464a-ab50-5e8e2381e5e0 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:08.336990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:08.437248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cc7ef-da47-464a-ab50-5e8e2381e5e0 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:08.475802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca75ab5-5073-48f2-8afb-4b4a60a8089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:08.476512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca75ab5-5073-48f2-8afb-4b4a60a8089b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.476862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca75ab5-5073-48f2-8afb-4b4a60a8089b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.477205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca75ab5-5073-48f2-8afb-4b4a60a8089b None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.477949 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca75ab5-5073-48f2-8afb-4b4a60a8089b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:08.478317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca75ab5-5073-48f2-8afb-4b4a60a8089b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:08.480162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca75ab5-5073-48f2-8afb-4b4a60a8089b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:08.492875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 237/480] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:09.606556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d623d-ea08-407f-82b7-1b11cb65800f None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:09.607431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d623d-ea08-407f-82b7-1b11cb65800f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:09.607431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d623d-ea08-407f-82b7-1b11cb65800f None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:09.608813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 244/481] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:09.614712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581ad2b-a2e8-41ac-a114-515992e22155 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:09.615652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581ad2b-a2e8-41ac-a114-515992e22155 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:09.615652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581ad2b-a2e8-41ac-a114-515992e22155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:09.641780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-3581ad2b-a2e8-41ac-a114-515992e22155 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:09.643005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3581ad2b-a2e8-41ac-a114-515992e22155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:09.743693 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 238/482] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:35:10.023642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d623d-ea08-407f-82b7-1b11cb65800f None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.104099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:10.184868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d623d-ea08-407f-82b7-1b11cb65800f None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.233775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae2af7-cf3a-4faf-b0b1-641fcbbaef4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:10.234494 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae2af7-cf3a-4faf-b0b1-641fcbbaef4c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.234837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae2af7-cf3a-4faf-b0b1-641fcbbaef4c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.235100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae2af7-cf3a-4faf-b0b1-641fcbbaef4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.256684 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae2af7-cf3a-4faf-b0b1-641fcbbaef4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-90ae2af7-cf3a-4faf-b0b1-641fcbbaef4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:10.262931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:10.264694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:10.267195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 245/483] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:10.275290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581ad2b-a2e8-41ac-a114-515992e22155 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.275695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:10.332403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4b897-e8c0-42c6-af2a-3f2900db6a0a None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.332862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4b897-e8c0-42c6-af2a-3f2900db6a0a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.333113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4b897-e8c0-42c6-af2a-3f2900db6a0a None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.334292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 246/484] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:10.341367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75021ddc-950c-4b59-8eb9-fadbaf4e730a None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.341728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75021ddc-950c-4b59-8eb9-fadbaf4e730a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.341904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75021ddc-950c-4b59-8eb9-fadbaf4e730a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.370382 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-75021ddc-950c-4b59-8eb9-fadbaf4e730a None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:10.371746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-75021ddc-950c-4b59-8eb9-fadbaf4e730a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:10.444744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581ad2b-a2e8-41ac-a114-515992e22155 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.478075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 247/485] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:35:10.502773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a541724-6814-465e-a490-b827572aad3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:10.503429 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a541724-6814-465e-a490-b827572aad3f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.503694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a541724-6814-465e-a490-b827572aad3f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.503944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a541724-6814-465e-a490-b827572aad3f None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.504611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a541724-6814-465e-a490-b827572aad3f None admin] RBAC: Adding query filter params (name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:10.504999 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a541724-6814-465e-a490-b827572aad3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:10.506362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a541724-6814-465e-a490-b827572aad3f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:10.517011 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 239/486] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:10.524252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75021ddc-950c-4b59-8eb9-fadbaf4e730a None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.593371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:10.668815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75021ddc-950c-4b59-8eb9-fadbaf4e730a None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.706700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b7e3b-e333-4157-9882-cb4a386871a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:10.707361 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b7e3b-e333-4157-9882-cb4a386871a3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.707640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b7e3b-e333-4157-9882-cb4a386871a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.707907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b7e3b-e333-4157-9882-cb4a386871a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.720021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b7e3b-e333-4157-9882-cb4a386871a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:10.721607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b7e3b-e333-4157-9882-cb4a386871a3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-290b7e3b-e333-4157-9882-cb4a386871a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:10.739703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:10.741949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:10.744061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:10.744061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:10.744061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:10.744061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 248/487] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:10.747048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a541724-6814-465e-a490-b827572aad3f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.812544 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b7e3b-e333-4157-9882-cb4a386871a3 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.846471 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:10.895599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a541724-6814-465e-a490-b827572aad3f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.937973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada7ac2-0e4f-4756-b67c-6c02bd9c21e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:10.938580 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada7ac2-0e4f-4756-b67c-6c02bd9c21e7 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.938856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada7ac2-0e4f-4756-b67c-6c02bd9c21e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.939111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada7ac2-0e4f-4756-b67c-6c02bd9c21e7 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.939799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada7ac2-0e4f-4756-b67c-6c02bd9c21e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:10.940095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada7ac2-0e4f-4756-b67c-6c02bd9c21e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:10.941504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada7ac2-0e4f-4756-b67c-6c02bd9c21e7 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:10.954610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 240/488] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:10.956030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064dec2-9141-4067-9bde-54664707b94d None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.956323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064dec2-9141-4067-9bde-54664707b94d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.956580 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064dec2-9141-4067-9bde-54664707b94d None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.963447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 241/489] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:10.963447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b226f82-7fff-4154-8001-77c960a8caf5 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.963447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b226f82-7fff-4154-8001-77c960a8caf5 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.963447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b226f82-7fff-4154-8001-77c960a8caf5 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.963447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 242/490] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:10.963447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b226f82-7fff-4154-8001-77c960a8caf5 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.984461 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:11.014980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b7e3b-e333-4157-9882-cb4a386871a3 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:11.086380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178efe9f-8ad7-4944-88cc-1efd3a6530fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.087024 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178efe9f-8ad7-4944-88cc-1efd3a6530fa None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.087393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178efe9f-8ad7-4944-88cc-1efd3a6530fa None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.087665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178efe9f-8ad7-4944-88cc-1efd3a6530fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.103392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178efe9f-8ad7-4944-88cc-1efd3a6530fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:11.107007 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b226f82-7fff-4154-8001-77c960a8caf5 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-178efe9f-8ad7-4944-88cc-1efd3a6530fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:11.113623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:11.115591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:11.117735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 249/491] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:11.119232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c5d76-724e-4b7c-a0fd-022fa6233127 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.119232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c5d76-724e-4b7c-a0fd-022fa6233127 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.119232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c5d76-724e-4b7c-a0fd-022fa6233127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.148719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-276c5d76-724e-4b7c-a0fd-022fa6233127 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:11.151873 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-276c5d76-724e-4b7c-a0fd-022fa6233127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:11.152775 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:11.164352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cec366-8ab2-4cf6-8955-9a322cc97fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.165010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cec366-8ab2-4cf6-8955-9a322cc97fa0 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.165279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cec366-8ab2-4cf6-8955-9a322cc97fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.165560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cec366-8ab2-4cf6-8955-9a322cc97fa0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.174803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cec366-8ab2-4cf6-8955-9a322cc97fa0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:11.176358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cec366-8ab2-4cf6-8955-9a322cc97fa0 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-89cec366-8ab2-4cf6-8955-9a322cc97fa0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:11.192015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:11.201424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 243/492] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6ed0b-5a13-4a84-b2f7-fcd92f52d8cb None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.203032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6ed0b-5a13-4a84-b2f7-fcd92f52d8cb None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.204350 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6ed0b-5a13-4a84-b2f7-fcd92f52d8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.220181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-9ee6ed0b-5a13-4a84-b2f7-fcd92f52d8cb None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:11.221701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9ee6ed0b-5a13-4a84-b2f7-fcd92f52d8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:11.288951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 250/493] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:11.291091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c5d76-724e-4b7c-a0fd-022fa6233127 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:11.360973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 244/494] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:11.362432 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6ed0b-5a13-4a84-b2f7-fcd92f52d8cb None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:11.521983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:11.556391 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c5d76-724e-4b7c-a0fd-022fa6233127 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:11.600746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6ed0b-5a13-4a84-b2f7-fcd92f52d8cb None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:11.609546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c44768-e36e-4d3e-9995-a07f9fb4a7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.610176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c44768-e36e-4d3e-9995-a07f9fb4a7d5 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.610457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c44768-e36e-4d3e-9995-a07f9fb4a7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.610733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c44768-e36e-4d3e-9995-a07f9fb4a7d5 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.611597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c44768-e36e-4d3e-9995-a07f9fb4a7d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:11.611842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c44768-e36e-4d3e-9995-a07f9fb4a7d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:11.613255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c44768-e36e-4d3e-9995-a07f9fb4a7d5 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:11.624956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 251/495] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:11.626284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f93922-7a6a-4913-ab88-eec0c99698ab None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.626572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f93922-7a6a-4913-ab88-eec0c99698ab None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.626863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f93922-7a6a-4913-ab88-eec0c99698ab None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.628552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 252/496] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:11.629410 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f93922-7a6a-4913-ab88-eec0c99698ab None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:11.634214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:11.658856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384cfad-ecda-449c-ad61-a5df2a68a13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.659519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384cfad-ecda-449c-ad61-a5df2a68a13a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.659792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384cfad-ecda-449c-ad61-a5df2a68a13a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.660060 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384cfad-ecda-449c-ad61-a5df2a68a13a None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.660815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384cfad-ecda-449c-ad61-a5df2a68a13a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:11.661119 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384cfad-ecda-449c-ad61-a5df2a68a13a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:11.662634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384cfad-ecda-449c-ad61-a5df2a68a13a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:11.712431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 245/497] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Aug 31 09:35:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:11.712431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384cfad-ecda-449c-ad61-a5df2a68a13a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:11.777287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f93922-7a6a-4913-ab88-eec0c99698ab None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:11.835883 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17f1ed-d65f-403e-b21d-9e9008d2b00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.836959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17f1ed-d65f-403e-b21d-9e9008d2b00c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.837446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17f1ed-d65f-403e-b21d-9e9008d2b00c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.837917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17f1ed-d65f-403e-b21d-9e9008d2b00c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.850191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:11.857561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17f1ed-d65f-403e-b21d-9e9008d2b00c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:11.857849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384cfad-ecda-449c-ad61-a5df2a68a13a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-0c17f1ed-d65f-403e-b21d-9e9008d2b00c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:11.864302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:11.865835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:11.867268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:11.867268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:11.867268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:11.867268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:11.867268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:11.867268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:11.867268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:11.867268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:11.867268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:11.867268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:11.867268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:11.867268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:11.868589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 253/498] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:11.870445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff0dbd-4955-476e-a456-f34b780018c3 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.870848 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff0dbd-4955-476e-a456-f34b780018c3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.871388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff0dbd-4955-476e-a456-f34b780018c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.889755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-efff0dbd-4955-476e-a456-f34b780018c3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:11.891269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-efff0dbd-4955-476e-a456-f34b780018c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:11.912204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86704091-44b9-4181-a49d-a14e8a53c83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.913323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86704091-44b9-4181-a49d-a14e8a53c83a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.913773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86704091-44b9-4181-a49d-a14e8a53c83a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.914245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86704091-44b9-4181-a49d-a14e8a53c83a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.935555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86704091-44b9-4181-a49d-a14e8a53c83a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:11.938062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86704091-44b9-4181-a49d-a14e8a53c83a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-86704091-44b9-4181-a49d-a14e8a53c83a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:11.957309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:11.959258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:11.960714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:11.960714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:11.960714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:11.960714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 246/499] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:11.960714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86704091-44b9-4181-a49d-a14e8a53c83a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:11.981658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 254/500] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:11.982536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff0dbd-4955-476e-a456-f34b780018c3 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.011141 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:12.132753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86704091-44b9-4181-a49d-a14e8a53c83a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.147581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff0dbd-4955-476e-a456-f34b780018c3 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.181414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb417cec-a74b-4562-b9e3-fe60651c6871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.182026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb417cec-a74b-4562-b9e3-fe60651c6871 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.182385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb417cec-a74b-4562-b9e3-fe60651c6871 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.183228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb417cec-a74b-4562-b9e3-fe60651c6871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.201984 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:12.218642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112ae6f-9041-436e-9817-dafc0294ed15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.219377 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112ae6f-9041-436e-9817-dafc0294ed15 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.219906 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112ae6f-9041-436e-9817-dafc0294ed15 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.220408 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112ae6f-9041-436e-9817-dafc0294ed15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.233716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112ae6f-9041-436e-9817-dafc0294ed15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-e112ae6f-9041-436e-9817-dafc0294ed15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:12.238536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:12.244359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 255/501] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:12.247857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112ae6f-9041-436e-9817-dafc0294ed15 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.312114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb417cec-a74b-4562-b9e3-fe60651c6871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.315000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb417cec-a74b-4562-b9e3-fe60651c6871 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.350717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:12.399581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112ae6f-9041-436e-9817-dafc0294ed15 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.434749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 247/502] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:12.435973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb417cec-a74b-4562-b9e3-fe60651c6871 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.460067 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98f6b0-3348-4e07-b3ba-aaca15f0a141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.461065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98f6b0-3348-4e07-b3ba-aaca15f0a141 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.461560 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98f6b0-3348-4e07-b3ba-aaca15f0a141 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.462042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98f6b0-3348-4e07-b3ba-aaca15f0a141 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.463442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98f6b0-3348-4e07-b3ba-aaca15f0a141 None admin] RBAC: Adding query filter params (name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:12.463916 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98f6b0-3348-4e07-b3ba-aaca15f0a141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.466489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98f6b0-3348-4e07-b3ba-aaca15f0a141 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.481725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 256/503] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:12.483051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98f6b0-3348-4e07-b3ba-aaca15f0a141 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.551450 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:12.553631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb417cec-a74b-4562-b9e3-fe60651c6871 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.583433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:12.595213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4d1c1-07f0-45bf-a683-9560ce6a4ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.596316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d1c1-07f0-45bf-a683-9560ce6a4ccf None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.596596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d1c1-07f0-45bf-a683-9560ce6a4ccf None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.596872 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d1c1-07f0-45bf-a683-9560ce6a4ccf None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.597544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4d1c1-07f0-45bf-a683-9560ce6a4ccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:12.597840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4d1c1-07f0-45bf-a683-9560ce6a4ccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.599293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4d1c1-07f0-45bf-a683-9560ce6a4ccf None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.612155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 248/504] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:12.613259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4d1c1-07f0-45bf-a683-9560ce6a4ccf None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.628659 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98f6b0-3348-4e07-b3ba-aaca15f0a141 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.669715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508b805-fafb-4301-9a83-0f3690fcf9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.670247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508b805-fafb-4301-9a83-0f3690fcf9cc None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.670531 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508b805-fafb-4301-9a83-0f3690fcf9cc None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.670807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508b805-fafb-4301-9a83-0f3690fcf9cc None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.671520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508b805-fafb-4301-9a83-0f3690fcf9cc None admin] RBAC: Adding query filter params (name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:12.671826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508b805-fafb-4301-9a83-0f3690fcf9cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.673388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508b805-fafb-4301-9a83-0f3690fcf9cc None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.683239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 257/505] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:12.684121 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508b805-fafb-4301-9a83-0f3690fcf9cc None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.769750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4d1c1-07f0-45bf-a683-9560ce6a4ccf None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.820703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508b805-fafb-4301-9a83-0f3690fcf9cc None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.827830 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ac108-5f97-4eec-a54d-cb36704eee5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.828904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ac108-5f97-4eec-a54d-cb36704eee5b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.829402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ac108-5f97-4eec-a54d-cb36704eee5b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.829866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ac108-5f97-4eec-a54d-cb36704eee5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.846320 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:12.850149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ac108-5f97-4eec-a54d-cb36704eee5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.852999 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ac108-5f97-4eec-a54d-cb36704eee5b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.872727 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6555f48-c169-44c6-a7d9-3cfc163b5033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.873347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6555f48-c169-44c6-a7d9-3cfc163b5033 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.873626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6555f48-c169-44c6-a7d9-3cfc163b5033 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.873900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6555f48-c169-44c6-a7d9-3cfc163b5033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-bd6ac108-5f97-4eec-a54d-cb36704eee5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:12.881242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:12.883359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:12.884427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:12.884427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:12.884427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:12.884427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 249/506] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:12.884745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ac108-5f97-4eec-a54d-cb36704eee5b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.891021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6555f48-c169-44c6-a7d9-3cfc163b5033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.892623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6555f48-c169-44c6-a7d9-3cfc163b5033 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-a6555f48-c169-44c6-a7d9-3cfc163b5033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:12.908676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:12.910565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:12.913001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:12.913001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:12.913001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:12.913001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 258/507] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:12.913001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6555f48-c169-44c6-a7d9-3cfc163b5033 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.951342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:13.038642 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ac108-5f97-4eec-a54d-cb36704eee5b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:13.039579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6555f48-c169-44c6-a7d9-3cfc163b5033 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:13.086010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4329e-239f-47c7-8455-ac3128c19988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:13.086659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4329e-239f-47c7-8455-ac3128c19988 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.086929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4329e-239f-47c7-8455-ac3128c19988 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.087238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4329e-239f-47c7-8455-ac3128c19988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.101066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4329e-239f-47c7-8455-ac3128c19988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:13.102023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834975a-2a1e-4cc2-b415-153666cd3872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:13.102630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4329e-239f-47c7-8455-ac3128c19988 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:13.103204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834975a-2a1e-4cc2-b415-153666cd3872 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.103713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834975a-2a1e-4cc2-b415-153666cd3872 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.104676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834975a-2a1e-4cc2-b415-153666cd3872 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-b8c4329e-239f-47c7-8455-ac3128c19988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:13.118958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:13.121517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:13.124246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:13.124246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:13.124246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:13.124645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834975a-2a1e-4cc2-b415-153666cd3872 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:13.124809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 250/508] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:13.124809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4329e-239f-47c7-8455-ac3128c19988 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:13.125082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834975a-2a1e-4cc2-b415-153666cd3872 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-9834975a-2a1e-4cc2-b415-153666cd3872 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:13.150167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:13.152115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:13.155474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:13.155474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:13.155474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:13.155474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:13.155474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:13.155474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:13.155474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:13.155474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:13.155474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:13.155474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:13.155474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 259/509] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:13.155474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834975a-2a1e-4cc2-b415-153666cd3872 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:13.199090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:13.248244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:13.263245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4329e-239f-47c7-8455-ac3128c19988 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:13.305327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e42688-2b4a-4053-ba0d-af40c3356bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:13.306169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e42688-2b4a-4053-ba0d-af40c3356bfe None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.306510 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e42688-2b4a-4053-ba0d-af40c3356bfe None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.306843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e42688-2b4a-4053-ba0d-af40c3356bfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.320170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834975a-2a1e-4cc2-b415-153666cd3872 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:13.322157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e42688-2b4a-4053-ba0d-af40c3356bfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:13.324886 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e42688-2b4a-4053-ba0d-af40c3356bfe None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-b8e42688-2b4a-4053-ba0d-af40c3356bfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:13.342612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:13.352460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:13.354306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:13.354306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:13.354306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:13.354306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 251/510] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:13.354306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e42688-2b4a-4053-ba0d-af40c3356bfe None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:13.363742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce6a29-2970-4e9a-87e3-8dcd4a282111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:13.364354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce6a29-2970-4e9a-87e3-8dcd4a282111 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.364631 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce6a29-2970-4e9a-87e3-8dcd4a282111 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.364901 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce6a29-2970-4e9a-87e3-8dcd4a282111 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.365564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce6a29-2970-4e9a-87e3-8dcd4a282111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:13.365862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce6a29-2970-4e9a-87e3-8dcd4a282111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:13.367303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce6a29-2970-4e9a-87e3-8dcd4a282111 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:13.387219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 260/511] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:13.388979 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce6a29-2970-4e9a-87e3-8dcd4a282111 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:13.488089 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e42688-2b4a-4053-ba0d-af40c3356bfe None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:13.526587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:13.535622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f946a-839e-4ede-b5d2-5fc56ee515a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:13.536582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f946a-839e-4ede-b5d2-5fc56ee515a1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.537045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f946a-839e-4ede-b5d2-5fc56ee515a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.537411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f946a-839e-4ede-b5d2-5fc56ee515a1 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.538156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f946a-839e-4ede-b5d2-5fc56ee515a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:13.538544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f946a-839e-4ede-b5d2-5fc56ee515a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:13.540148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f946a-839e-4ede-b5d2-5fc56ee515a1 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:13.547860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce6a29-2970-4e9a-87e3-8dcd4a282111 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:13.556002 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 252/512] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:13.558160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f946a-839e-4ede-b5d2-5fc56ee515a1 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:13.562570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:13.596838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5cc24-40d3-442d-a562-416571bf9c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:13.597567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5cc24-40d3-442d-a562-416571bf9c9c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.597925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5cc24-40d3-442d-a562-416571bf9c9c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.598308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5cc24-40d3-442d-a562-416571bf9c9c None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.599092 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5cc24-40d3-442d-a562-416571bf9c9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:13.599508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5cc24-40d3-442d-a562-416571bf9c9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:13.601087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5cc24-40d3-442d-a562-416571bf9c9c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:13.614664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 261/513] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:13.615695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5cc24-40d3-442d-a562-416571bf9c9c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:13.690392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f946a-839e-4ede-b5d2-5fc56ee515a1 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:13.730773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6d4fa-f159-41e1-ae02-4e213aae45cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:13.731951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6d4fa-f159-41e1-ae02-4e213aae45cf None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.732487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6d4fa-f159-41e1-ae02-4e213aae45cf None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.732958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6d4fa-f159-41e1-ae02-4e213aae45cf None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.734352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6d4fa-f159-41e1-ae02-4e213aae45cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:13.734887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6d4fa-f159-41e1-ae02-4e213aae45cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:13.737714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6d4fa-f159-41e1-ae02-4e213aae45cf None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:13.759563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5cc24-40d3-442d-a562-416571bf9c9c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:13.774078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 253/514] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 09:35:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:13.775643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6d4fa-f159-41e1-ae02-4e213aae45cf None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:13.798070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4946529-1b03-4ab7-a99f-abb0af841145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:13.798666 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4946529-1b03-4ab7-a99f-abb0af841145 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.799021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4946529-1b03-4ab7-a99f-abb0af841145 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.799333 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4946529-1b03-4ab7-a99f-abb0af841145 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.800005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4946529-1b03-4ab7-a99f-abb0af841145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:13.800299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4946529-1b03-4ab7-a99f-abb0af841145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:13.801691 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4946529-1b03-4ab7-a99f-abb0af841145 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:13.815624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 262/515] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:13.816536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4946529-1b03-4ab7-a99f-abb0af841145 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:13.825537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:13.867234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:13.921949 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6d4fa-f159-41e1-ae02-4e213aae45cf None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:13.946894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4946529-1b03-4ab7-a99f-abb0af841145 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:13.965511 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42a07c-48c9-4d9c-8869-8eb87d5c7314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:13.966244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42a07c-48c9-4d9c-8869-8eb87d5c7314 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.966580 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42a07c-48c9-4d9c-8869-8eb87d5c7314 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.966970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42a07c-48c9-4d9c-8869-8eb87d5c7314 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.967854 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42a07c-48c9-4d9c-8869-8eb87d5c7314 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:13.969424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42a07c-48c9-4d9c-8869-8eb87d5c7314 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:13.989227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c9d3c-19f6-4d9f-ac4b-676dd8c61178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:13.990147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c9d3c-19f6-4d9f-ac4b-676dd8c61178 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.990601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c9d3c-19f6-4d9f-ac4b-676dd8c61178 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.991089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c9d3c-19f6-4d9f-ac4b-676dd8c61178 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.007283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 254/516] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:35:13 2022] POST /identity/v3/users => generated 247 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:14.007283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42a07c-48c9-4d9c-8869-8eb87d5c7314 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.007671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c9d3c-19f6-4d9f-ac4b-676dd8c61178 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.007671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c9d3c-19f6-4d9f-ac4b-676dd8c61178 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-a81c9d3c-19f6-4d9f-ac4b-676dd8c61178 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:14.019574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:14.021298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:14.023635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:14.023635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:14.023635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:14.023635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:14.023635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:14.023635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:14.023635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:14.023635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:14.023635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 263/517] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:14.023635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c9d3c-19f6-4d9f-ac4b-676dd8c61178 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.150537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:14.194194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42a07c-48c9-4d9c-8869-8eb87d5c7314 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.208766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c9d3c-19f6-4d9f-ac4b-676dd8c61178 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.218650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:14.259153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74f7b5-2dc8-4b8d-9515-fdc4458ab296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.259795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74f7b5-2dc8-4b8d-9515-fdc4458ab296 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.260074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74f7b5-2dc8-4b8d-9515-fdc4458ab296 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.260401 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74f7b5-2dc8-4b8d-9515-fdc4458ab296 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.267710 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2caa7-80ab-4a6a-b2f2-b0ec28f0d4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.268646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2caa7-80ab-4a6a-b2f2-b0ec28f0d4f9 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.269090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2caa7-80ab-4a6a-b2f2-b0ec28f0d4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.269539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2caa7-80ab-4a6a-b2f2-b0ec28f0d4f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.273990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74f7b5-2dc8-4b8d-9515-fdc4458ab296 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.277633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74f7b5-2dc8-4b8d-9515-fdc4458ab296 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-8f74f7b5-2dc8-4b8d-9515-fdc4458ab296 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:14.300149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:14.301973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:14.303813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:14.303813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:14.303813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:14.303813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:14.303813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:14.303813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:14.303813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:14.303813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:14.303813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:35:14.303813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:14.303813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 255/518] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:14.303813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74f7b5-2dc8-4b8d-9515-fdc4458ab296 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.387389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2caa7-80ab-4a6a-b2f2-b0ec28f0d4f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.389322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2caa7-80ab-4a6a-b2f2-b0ec28f0d4f9 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.470991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74f7b5-2dc8-4b8d-9515-fdc4458ab296 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.474043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:14.533486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 264/519] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:14.533486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2caa7-80ab-4a6a-b2f2-b0ec28f0d4f9 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.541335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b08d3-873d-42f6-bf93-84874c5731f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.543073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b08d3-873d-42f6-bf93-84874c5731f1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.543589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b08d3-873d-42f6-bf93-84874c5731f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.544047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b08d3-873d-42f6-bf93-84874c5731f1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.558397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b08d3-873d-42f6-bf93-84874c5731f1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.561448 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b08d3-873d-42f6-bf93-84874c5731f1 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-336b08d3-873d-42f6-bf93-84874c5731f1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:14.583025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:14.584101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:14.584977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:14.584977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:14.584977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:14.584977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:14.584977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:14.584977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:14.584977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:14.584977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:14.584977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:35:14.584977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:14.593077 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 256/520] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:14.594609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b08d3-873d-42f6-bf93-84874c5731f1 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.642075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:14.703425 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2caa7-80ab-4a6a-b2f2-b0ec28f0d4f9 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.723650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b08d3-873d-42f6-bf93-84874c5731f1 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.761684 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3c23b-3e18-4641-bdc9-ffb65d96049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.762750 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3c23b-3e18-4641-bdc9-ffb65d96049f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.763282 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3c23b-3e18-4641-bdc9-ffb65d96049f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.763764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3c23b-3e18-4641-bdc9-ffb65d96049f None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.765368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3c23b-3e18-4641-bdc9-ffb65d96049f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:14.765912 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3c23b-3e18-4641-bdc9-ffb65d96049f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.768335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670f410-3b17-44ad-bcaf-7756f5d8f72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.768694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3c23b-3e18-4641-bdc9-ffb65d96049f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.768932 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670f410-3b17-44ad-bcaf-7756f5d8f72f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.769221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670f410-3b17-44ad-bcaf-7756f5d8f72f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.769487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670f410-3b17-44ad-bcaf-7756f5d8f72f None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.770228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670f410-3b17-44ad-bcaf-7756f5d8f72f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:14.770529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670f410-3b17-44ad-bcaf-7756f5d8f72f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.772315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670f410-3b17-44ad-bcaf-7756f5d8f72f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.802458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 257/521] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Aug 31 09:35:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:14.802458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670f410-3b17-44ad-bcaf-7756f5d8f72f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.802847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 265/522] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:14.805043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3c23b-3e18-4641-bdc9-ffb65d96049f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.840382 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:14.940316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3c23b-3e18-4641-bdc9-ffb65d96049f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.985490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323c460-aa06-4f5e-ac2a-357ba9d3b8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.986109 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323c460-aa06-4f5e-ac2a-357ba9d3b8e9 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.986384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323c460-aa06-4f5e-ac2a-357ba9d3b8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.986659 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323c460-aa06-4f5e-ac2a-357ba9d3b8e9 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.987445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323c460-aa06-4f5e-ac2a-357ba9d3b8e9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:14.987747 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323c460-aa06-4f5e-ac2a-357ba9d3b8e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.989243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323c460-aa06-4f5e-ac2a-357ba9d3b8e9 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.993428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:15.010194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670f410-3b17-44ad-bcaf-7756f5d8f72f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.034630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 266/523] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Aug 31 09:35:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:15.036070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323c460-aa06-4f5e-ac2a-357ba9d3b8e9 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.071092 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0a93a-edab-40f2-9d7e-9ce0227efa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:15.075272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0a93a-edab-40f2-9d7e-9ce0227efa23 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.075272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0a93a-edab-40f2-9d7e-9ce0227efa23 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.075272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0a93a-edab-40f2-9d7e-9ce0227efa23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.090151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0a93a-edab-40f2-9d7e-9ce0227efa23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:15.092572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0a93a-edab-40f2-9d7e-9ce0227efa23 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-69e0a93a-edab-40f2-9d7e-9ce0227efa23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:15.115295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:15.116524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:15.117544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:15.117544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:15.117544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:15.123504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 258/524] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:15.124905 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0a93a-edab-40f2-9d7e-9ce0227efa23 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.165696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:15.191281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323c460-aa06-4f5e-ac2a-357ba9d3b8e9 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.236014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfd1e1-1418-4c27-b18c-5db69b9a029e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:15.236635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfd1e1-1418-4c27-b18c-5db69b9a029e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.236911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfd1e1-1418-4c27-b18c-5db69b9a029e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.237182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfd1e1-1418-4c27-b18c-5db69b9a029e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.253297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0a93a-edab-40f2-9d7e-9ce0227efa23 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.295397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f37ad-304d-4cf5-98d6-f0f5f4819856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:15.296001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f37ad-304d-4cf5-98d6-f0f5f4819856 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.296276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f37ad-304d-4cf5-98d6-f0f5f4819856 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.296541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f37ad-304d-4cf5-98d6-f0f5f4819856 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.297279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f37ad-304d-4cf5-98d6-f0f5f4819856 None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=06fad9466df14827983dac2b4b844f3a) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:15.297626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f37ad-304d-4cf5-98d6-f0f5f4819856 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:15.299043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f37ad-304d-4cf5-98d6-f0f5f4819856 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:15.311583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 259/525] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/role_assignments?user.id=06fad9466df14827983dac2b4b844f3a&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:15.312968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f37ad-304d-4cf5-98d6-f0f5f4819856 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.336191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:15.341346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfd1e1-1418-4c27-b18c-5db69b9a029e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:15.346208 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfd1e1-1418-4c27-b18c-5db69b9a029e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:15.496242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:15.513645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f37ad-304d-4cf5-98d6-f0f5f4819856 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.538228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 267/526] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:15.538228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfd1e1-1418-4c27-b18c-5db69b9a029e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.572284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478613a-91ea-4bd0-9088-1683402a0270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:15.573630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478613a-91ea-4bd0-9088-1683402a0270 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.574240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478613a-91ea-4bd0-9088-1683402a0270 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.574240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478613a-91ea-4bd0-9088-1683402a0270 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.652305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478613a-91ea-4bd0-9088-1683402a0270 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:15.653441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478613a-91ea-4bd0-9088-1683402a0270 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:15.667086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:15.694035 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfd1e1-1418-4c27-b18c-5db69b9a029e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.736658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f5558-2fb9-48b5-a566-c19ede12b49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:15.737675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f5558-2fb9-48b5-a566-c19ede12b49c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.738176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f5558-2fb9-48b5-a566-c19ede12b49c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.738664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f5558-2fb9-48b5-a566-c19ede12b49c None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.739785 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f5558-2fb9-48b5-a566-c19ede12b49c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:15.740294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f5558-2fb9-48b5-a566-c19ede12b49c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:15.742008 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f5558-2fb9-48b5-a566-c19ede12b49c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:15.756892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 268/527] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:15.758272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f5558-2fb9-48b5-a566-c19ede12b49c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.790519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 260/528] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:15.790519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478613a-91ea-4bd0-9088-1683402a0270 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.827217 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:15.896056 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f5558-2fb9-48b5-a566-c19ede12b49c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.903860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478613a-91ea-4bd0-9088-1683402a0270 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.915804 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:15.941541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72738a5-7e63-44b2-b6eb-07903407132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:15.942554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72738a5-7e63-44b2-b6eb-07903407132a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.943047 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72738a5-7e63-44b2-b6eb-07903407132a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.943335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a81da9-68b6-48ab-b4e6-fc4e385d086f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:15.943831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a81da9-68b6-48ab-b4e6-fc4e385d086f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.944048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72738a5-7e63-44b2-b6eb-07903407132a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.944145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a81da9-68b6-48ab-b4e6-fc4e385d086f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.944362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a81da9-68b6-48ab-b4e6-fc4e385d086f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.952089 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a81da9-68b6-48ab-b4e6-fc4e385d086f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:15.953939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a81da9-68b6-48ab-b4e6-fc4e385d086f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:15.958413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72738a5-7e63-44b2-b6eb-07903407132a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:15.960223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72738a5-7e63-44b2-b6eb-07903407132a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-87a81da9-68b6-48ab-b4e6-fc4e385d086f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:15.966971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:15.967946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:15.968779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:15.968779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:15.968779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:15.968779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:15.968779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:15.968779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:15.968779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:15.968779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:15.973060 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 261/529] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:15.974054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a81da9-68b6-48ab-b4e6-fc4e385d086f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-e72738a5-7e63-44b2-b6eb-07903407132a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:15.982838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:15.983900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:15.984868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:15.984868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:15.984868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:15.991470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 269/530] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:15.992798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72738a5-7e63-44b2-b6eb-07903407132a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.109884 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a81da9-68b6-48ab-b4e6-fc4e385d086f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.124781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:16.134821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72738a5-7e63-44b2-b6eb-07903407132a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.167753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1a68d-1084-4e1a-b5b6-a890f0e8026b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.168807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1a68d-1084-4e1a-b5b6-a890f0e8026b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.169090 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1a68d-1084-4e1a-b5b6-a890f0e8026b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.169359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1a68d-1084-4e1a-b5b6-a890f0e8026b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.183671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2f41a-f8a2-445a-927b-f496c985ccd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.184622 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2f41a-f8a2-445a-927b-f496c985ccd4 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.185076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2f41a-f8a2-445a-927b-f496c985ccd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.185496 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2f41a-f8a2-445a-927b-f496c985ccd4 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.186423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1a68d-1084-4e1a-b5b6-a890f0e8026b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.186636 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2f41a-f8a2-445a-927b-f496c985ccd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:16.187091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2f41a-f8a2-445a-927b-f496c985ccd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.187944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1a68d-1084-4e1a-b5b6-a890f0e8026b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.188975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2f41a-f8a2-445a-927b-f496c985ccd4 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.204214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 270/531] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:16.205520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2f41a-f8a2-445a-927b-f496c985ccd4 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-98f1a68d-1084-4e1a-b5b6-a890f0e8026b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:16.210994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:16.212341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:16.213179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:16.213179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:16.213179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:16.217675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 262/532] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:16.218740 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1a68d-1084-4e1a-b5b6-a890f0e8026b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.249957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:16.344380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1a68d-1084-4e1a-b5b6-a890f0e8026b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.352085 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2f41a-f8a2-445a-927b-f496c985ccd4 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.384079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc13d11-7837-4c37-8f6f-e41d7906994f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.384654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc13d11-7837-4c37-8f6f-e41d7906994f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.384930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc13d11-7837-4c37-8f6f-e41d7906994f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.386085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc13d11-7837-4c37-8f6f-e41d7906994f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.387257 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc13d11-7837-4c37-8f6f-e41d7906994f None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=ef72bb95992c4a21ae2dbb2a08c51421) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:16.387525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc13d11-7837-4c37-8f6f-e41d7906994f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.389015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc13d11-7837-4c37-8f6f-e41d7906994f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.403171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 263/533] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:16 2022] GET /identity/v3/role_assignments?user.id=ef72bb95992c4a21ae2dbb2a08c51421&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:16.403171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc13d11-7837-4c37-8f6f-e41d7906994f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.406101 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bb04b-b277-4252-845c-bd1b3f157cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.407012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bb04b-b277-4252-845c-bd1b3f157cae None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.407467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bb04b-b277-4252-845c-bd1b3f157cae None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.407866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bb04b-b277-4252-845c-bd1b3f157cae None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.408698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bb04b-b277-4252-845c-bd1b3f157cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:16.409194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bb04b-b277-4252-845c-bd1b3f157cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.410771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bb04b-b277-4252-845c-bd1b3f157cae None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.425558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 271/534] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:16.434254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bb04b-b277-4252-845c-bd1b3f157cae None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.471900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:16.618225 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:16.640646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc13d11-7837-4c37-8f6f-e41d7906994f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.667684 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bb04b-b277-4252-845c-bd1b3f157cae None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.693409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bbb85-a42a-4953-bddd-65754a215dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.694059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bbb85-a42a-4953-bddd-65754a215dbb None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.694341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bbb85-a42a-4953-bddd-65754a215dbb None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.695845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bbb85-a42a-4953-bddd-65754a215dbb None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.695845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bbb85-a42a-4953-bddd-65754a215dbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:16.695845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bbb85-a42a-4953-bddd-65754a215dbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.697047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bbb85-a42a-4953-bddd-65754a215dbb None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.722005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 264/535] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:16.728103 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bbb85-a42a-4953-bddd-65754a215dbb None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.745205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db53da-191c-438e-87f5-99d4f37322a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.746713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db53da-191c-438e-87f5-99d4f37322a7 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.747448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db53da-191c-438e-87f5-99d4f37322a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.748189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db53da-191c-438e-87f5-99d4f37322a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.771568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db53da-191c-438e-87f5-99d4f37322a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.774613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db53da-191c-438e-87f5-99d4f37322a7 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-e0db53da-191c-438e-87f5-99d4f37322a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:16.813303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:16.814823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:16.816548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:16.816548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:16.816548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:16.816548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:16.816548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:16.816548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:16.816548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:16.816548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:16.828765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 272/536] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:16.838085 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db53da-191c-438e-87f5-99d4f37322a7 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.861702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bbb85-a42a-4953-bddd-65754a215dbb None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.903770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83adaa97-7b5d-4867-a896-86edf02e9810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.904307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adaa97-7b5d-4867-a896-86edf02e9810 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.904574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adaa97-7b5d-4867-a896-86edf02e9810 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.905022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adaa97-7b5d-4867-a896-86edf02e9810 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.919359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83adaa97-7b5d-4867-a896-86edf02e9810 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.920844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83adaa97-7b5d-4867-a896-86edf02e9810 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-83adaa97-7b5d-4867-a896-86edf02e9810 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:16.944414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:16.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:16.947891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:16.947891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:16.947891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:16.947891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:16.947891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:16.947891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:16.947891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:16.947891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:16.947891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 265/537] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:16.953084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83adaa97-7b5d-4867-a896-86edf02e9810 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.995554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:17.079221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:17.131521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db53da-191c-438e-87f5-99d4f37322a7 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:17.190778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83adaa97-7b5d-4867-a896-86edf02e9810 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:17.200102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc4a0d-f8e1-4832-b03a-6f02d7ff1774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:17.201127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc4a0d-f8e1-4832-b03a-6f02d7ff1774 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:17.201616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc4a0d-f8e1-4832-b03a-6f02d7ff1774 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:17.202093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc4a0d-f8e1-4832-b03a-6f02d7ff1774 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:17.203367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc4a0d-f8e1-4832-b03a-6f02d7ff1774 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:17.203917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc4a0d-f8e1-4832-b03a-6f02d7ff1774 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:17.206673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc4a0d-f8e1-4832-b03a-6f02d7ff1774 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:17.240009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 273/538] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:17.242580 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7247214-ecaa-4c06-b476-61d13a40168a None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:17.243040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7247214-ecaa-4c06-b476-61d13a40168a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:17.243682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7247214-ecaa-4c06-b476-61d13a40168a None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:17.248544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bde706-47df-46a6-aac7-8d6641b4c99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:17.249040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 274/539] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:17.249209 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bde706-47df-46a6-aac7-8d6641b4c99b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:17.249527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bde706-47df-46a6-aac7-8d6641b4c99b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:17.249694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bde706-47df-46a6-aac7-8d6641b4c99b None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:17.250380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bde706-47df-46a6-aac7-8d6641b4c99b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:17.250550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7247214-ecaa-4c06-b476-61d13a40168a None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:17.250699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bde706-47df-46a6-aac7-8d6641b4c99b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:17.252458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bde706-47df-46a6-aac7-8d6641b4c99b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:17.278045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 266/540] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:17.279737 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673abcc-c2e9-4fb5-833e-eb58f01b5cc9 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:17.280074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673abcc-c2e9-4fb5-833e-eb58f01b5cc9 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:17.280347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673abcc-c2e9-4fb5-833e-eb58f01b5cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:17.303937 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-5673abcc-c2e9-4fb5-833e-eb58f01b5cc9 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:17.305667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5673abcc-c2e9-4fb5-833e-eb58f01b5cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:17.394078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:17.405121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 267/541] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:17.406496 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673abcc-c2e9-4fb5-833e-eb58f01b5cc9 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:17.419649 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7247214-ecaa-4c06-b476-61d13a40168a None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:17.475826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd9cdc-5efb-43f1-a606-6e707d1066bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:17.476469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd9cdc-5efb-43f1-a606-6e707d1066bd None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:17.476752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd9cdc-5efb-43f1-a606-6e707d1066bd None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:17.477022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd9cdc-5efb-43f1-a606-6e707d1066bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:17.477767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd9cdc-5efb-43f1-a606-6e707d1066bd None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=6ff1177119cb48e0a88b0f11a6b1b0c3) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:17.478074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd9cdc-5efb-43f1-a606-6e707d1066bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:17.481583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd9cdc-5efb-43f1-a606-6e707d1066bd None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:17.484492 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:17.495218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 275/542] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:17 2022] GET /identity/v3/role_assignments?user.id=6ff1177119cb48e0a88b0f11a6b1b0c3&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:17.551574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673abcc-c2e9-4fb5-833e-eb58f01b5cc9 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:17.597298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fd095-9bac-47b4-9eb5-bb5c32d4be3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:17.598385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fd095-9bac-47b4-9eb5-bb5c32d4be3f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:17.598856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fd095-9bac-47b4-9eb5-bb5c32d4be3f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:17.599355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fd095-9bac-47b4-9eb5-bb5c32d4be3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:17.600300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fd095-9bac-47b4-9eb5-bb5c32d4be3f None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=21d8ade66b7e4a1880c961c5e13adf04) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:17.600807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fd095-9bac-47b4-9eb5-bb5c32d4be3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:17.602501 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fd095-9bac-47b4-9eb5-bb5c32d4be3f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:17.610098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 268/543] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:17 2022] GET /identity/v3/role_assignments?user.id=21d8ade66b7e4a1880c961c5e13adf04&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:17.980345 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd9cdc-5efb-43f1-a606-6e707d1066bd None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:17.991212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f1d33-6a44-414d-93ea-cf0668691615 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:17.991519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f1d33-6a44-414d-93ea-cf0668691615 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:17.991783 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f1d33-6a44-414d-93ea-cf0668691615 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:17.993395 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 269/544] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:18.000217 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15aa34-9410-4a00-8636-d93db16891ea None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:18.000485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15aa34-9410-4a00-8636-d93db16891ea None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:18.000755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15aa34-9410-4a00-8636-d93db16891ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:18.026400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-1a15aa34-9410-4a00-8636-d93db16891ea None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:18.028450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1a15aa34-9410-4a00-8636-d93db16891ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:18.177754 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:18.212138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 270/545] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:35:18.307858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd9cdc-5efb-43f1-a606-6e707d1066bd None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:18.390346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0023c6-d307-48dc-807e-eddda6fcb04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:18.391709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0023c6-d307-48dc-807e-eddda6fcb04a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:18.392340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0023c6-d307-48dc-807e-eddda6fcb04a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:18.392956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0023c6-d307-48dc-807e-eddda6fcb04a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:18.424687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0023c6-d307-48dc-807e-eddda6fcb04a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-8e0023c6-d307-48dc-807e-eddda6fcb04a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:18.435895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:18.437779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:18.443757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:18.443757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:18.443757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:18.443757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:18.443757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:18.443757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:18.443757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:18.443757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:18.443757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:18.443757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:18.443757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:18.443757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:18.447466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 276/546] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 469 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:18.457372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15aa34-9410-4a00-8636-d93db16891ea None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:18.665030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0023c6-d307-48dc-807e-eddda6fcb04a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:18.678604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15aa34-9410-4a00-8636-d93db16891ea None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:18.746208 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc6d2e-0d75-44d8-8653-bf5e0bb3a9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:18.747185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc6d2e-0d75-44d8-8653-bf5e0bb3a9c8 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:18.747504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc6d2e-0d75-44d8-8653-bf5e0bb3a9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:18.747779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc6d2e-0d75-44d8-8653-bf5e0bb3a9c8 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:18.748474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc6d2e-0d75-44d8-8653-bf5e0bb3a9c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:18.748775 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc6d2e-0d75-44d8-8653-bf5e0bb3a9c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:18.750199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc6d2e-0d75-44d8-8653-bf5e0bb3a9c8 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:18.768847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 271/547] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:18.774175 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc6d2e-0d75-44d8-8653-bf5e0bb3a9c8 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:18.777731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:18.833594 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0023c6-d307-48dc-807e-eddda6fcb04a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:18.874323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf204a-54de-4f12-b89a-298888392896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:18.874970 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf204a-54de-4f12-b89a-298888392896 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:18.875291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf204a-54de-4f12-b89a-298888392896 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:18.875941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf204a-54de-4f12-b89a-298888392896 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:18.890567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:18.895421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf204a-54de-4f12-b89a-298888392896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-9abf204a-54de-4f12-b89a-298888392896 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:18.901508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:18.903583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:18.904940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 277/548] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:18.913226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf204a-54de-4f12-b89a-298888392896 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:18.997981 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc6d2e-0d75-44d8-8653-bf5e0bb3a9c8 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.059172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de0cd6-57cc-4438-a4db-1856833c2ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.059172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de0cd6-57cc-4438-a4db-1856833c2ea5 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.059172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de0cd6-57cc-4438-a4db-1856833c2ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.059172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de0cd6-57cc-4438-a4db-1856833c2ea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.091152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de0cd6-57cc-4438-a4db-1856833c2ea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.091152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de0cd6-57cc-4438-a4db-1856833c2ea5 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-35de0cd6-57cc-4438-a4db-1856833c2ea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:19.136403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:19.138437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:19.140417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:19.140417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:19.140417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:19.153765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 272/549] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:19.160905 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de0cd6-57cc-4438-a4db-1856833c2ea5 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.211295 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf204a-54de-4f12-b89a-298888392896 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.239408 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:19.301095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66fee0-5a4c-4a10-8046-2ac7007820a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.302020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66fee0-5a4c-4a10-8046-2ac7007820a4 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.302448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66fee0-5a4c-4a10-8046-2ac7007820a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.302886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66fee0-5a4c-4a10-8046-2ac7007820a4 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.303764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66fee0-5a4c-4a10-8046-2ac7007820a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:19.304220 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66fee0-5a4c-4a10-8046-2ac7007820a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.311156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66fee0-5a4c-4a10-8046-2ac7007820a4 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.326084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de0cd6-57cc-4438-a4db-1856833c2ea5 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.328695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:19.330067 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 278/550] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:19.341096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66fee0-5a4c-4a10-8046-2ac7007820a4 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.375322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc970e-60b5-4ea6-bb0e-2df65f9824c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.375943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc970e-60b5-4ea6-bb0e-2df65f9824c3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.376214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc970e-60b5-4ea6-bb0e-2df65f9824c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.376476 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc970e-60b5-4ea6-bb0e-2df65f9824c3 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.377124 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc970e-60b5-4ea6-bb0e-2df65f9824c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:19.377421 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc970e-60b5-4ea6-bb0e-2df65f9824c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.378812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc970e-60b5-4ea6-bb0e-2df65f9824c3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.406457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 273/551] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:19.413316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc970e-60b5-4ea6-bb0e-2df65f9824c3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.544078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc970e-60b5-4ea6-bb0e-2df65f9824c3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.576576 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66fee0-5a4c-4a10-8046-2ac7007820a4 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.593071 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117dcdf-3d39-4ea7-896f-28f5336eb156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.593725 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117dcdf-3d39-4ea7-896f-28f5336eb156 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.594036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117dcdf-3d39-4ea7-896f-28f5336eb156 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.594266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117dcdf-3d39-4ea7-896f-28f5336eb156 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.602857 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117dcdf-3d39-4ea7-896f-28f5336eb156 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.604841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117dcdf-3d39-4ea7-896f-28f5336eb156 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-9117dcdf-3d39-4ea7-896f-28f5336eb156 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:19.617507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:19.618512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:19.620008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:19.620008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:19.620008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:19.620008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:19.620008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:19.620008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:19.620008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:19.620008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:19.620008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:19.620008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:19.620008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 274/552] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:19.621115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fa2d9-5481-4ebd-ac69-4707332350d4 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.623114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fa2d9-5481-4ebd-ac69-4707332350d4 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.623114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fa2d9-5481-4ebd-ac69-4707332350d4 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.626596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 275/553] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:19.627646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fa2d9-5481-4ebd-ac69-4707332350d4 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.655947 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164bed3-3569-41df-a459-02f17763bc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.656870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164bed3-3569-41df-a459-02f17763bc37 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.657323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164bed3-3569-41df-a459-02f17763bc37 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.657748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164bed3-3569-41df-a459-02f17763bc37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.671417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164bed3-3569-41df-a459-02f17763bc37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.673206 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164bed3-3569-41df-a459-02f17763bc37 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-9164bed3-3569-41df-a459-02f17763bc37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:19.689857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:19.691222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:19.692306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:19.692306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:19.692306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:19.692484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:19.699593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 279/554] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:19.702472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010858c-5af9-4504-bae2-a294edf61bad None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.703254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010858c-5af9-4504-bae2-a294edf61bad None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.703989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010858c-5af9-4504-bae2-a294edf61bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.729923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-8010858c-5af9-4504-bae2-a294edf61bad None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:19.732407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-8010858c-5af9-4504-bae2-a294edf61bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:19.769011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:19.827981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 280/555] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:19.834394 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010858c-5af9-4504-bae2-a294edf61bad None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.835291 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fa2d9-5481-4ebd-ac69-4707332350d4 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.903225 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec88aa0-5249-4b7b-ace1-286b1157d3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.903900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec88aa0-5249-4b7b-ace1-286b1157d3f5 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.904177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec88aa0-5249-4b7b-ace1-286b1157d3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.904478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec88aa0-5249-4b7b-ace1-286b1157d3f5 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.905470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec88aa0-5249-4b7b-ace1-286b1157d3f5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:19.905851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec88aa0-5249-4b7b-ace1-286b1157d3f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.910284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec88aa0-5249-4b7b-ace1-286b1157d3f5 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.958243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 276/556] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Aug 31 09:35:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:19.967581 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec88aa0-5249-4b7b-ace1-286b1157d3f5 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:20.113173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010858c-5af9-4504-bae2-a294edf61bad None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.139997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:20.179798 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec88aa0-5249-4b7b-ace1-286b1157d3f5 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.192447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb6bbd-8288-4c10-8b7e-9c9377509dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.193060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb6bbd-8288-4c10-8b7e-9c9377509dfe None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.193335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb6bbd-8288-4c10-8b7e-9c9377509dfe None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.193604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb6bbd-8288-4c10-8b7e-9c9377509dfe None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.194255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb6bbd-8288-4c10-8b7e-9c9377509dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:20.194557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb6bbd-8288-4c10-8b7e-9c9377509dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:20.196052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb6bbd-8288-4c10-8b7e-9c9377509dfe None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:20.214364 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 281/557] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:20.215908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594ec99-10a3-40a1-9131-1a9846beba23 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.216297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594ec99-10a3-40a1-9131-1a9846beba23 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.216608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594ec99-10a3-40a1-9131-1a9846beba23 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.218239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 282/558] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:20.219047 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594ec99-10a3-40a1-9131-1a9846beba23 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:20.252448 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ab4da-4500-4b93-a468-80dca780028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.253075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ab4da-4500-4b93-a468-80dca780028a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.253349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ab4da-4500-4b93-a468-80dca780028a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.253643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ab4da-4500-4b93-a468-80dca780028a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.256832 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:20.351190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ab4da-4500-4b93-a468-80dca780028a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:20.353295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ab4da-4500-4b93-a468-80dca780028a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:20.400744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594ec99-10a3-40a1-9131-1a9846beba23 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.441201 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736854-f31a-4b34-861e-c20fd5737591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.441857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736854-f31a-4b34-861e-c20fd5737591 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.442133 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736854-f31a-4b34-861e-c20fd5737591 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.442398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736854-f31a-4b34-861e-c20fd5737591 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.455374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f736854-f31a-4b34-861e-c20fd5737591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-7f736854-f31a-4b34-861e-c20fd5737591 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:20.460922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:20.461918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:20.462744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 283/559] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:20.464888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736854-f31a-4b34-861e-c20fd5737591 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:20.501952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:20.534208 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 277/560] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:20.536223 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065a530-363e-402c-858e-6af42ce45f55 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.536802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065a530-363e-402c-858e-6af42ce45f55 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.537278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065a530-363e-402c-858e-6af42ce45f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.562148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-5065a530-363e-402c-858e-6af42ce45f55 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:20.563469 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5065a530-363e-402c-858e-6af42ce45f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:20.580919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:20.609792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736854-f31a-4b34-861e-c20fd5737591 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.643882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 278/561] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:20.645195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e60d9-4ef5-4312-9867-472635e07147 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.645477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e60d9-4ef5-4312-9867-472635e07147 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.645747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e60d9-4ef5-4312-9867-472635e07147 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.647051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 279/562] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:20.652621 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e60d9-4ef5-4312-9867-472635e07147 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:20.679986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05cc5c-cb45-4537-9644-15b4858df44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.681063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05cc5c-cb45-4537-9644-15b4858df44c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.681550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05cc5c-cb45-4537-9644-15b4858df44c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.682045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05cc5c-cb45-4537-9644-15b4858df44c None admin] PATH_INFO: `/v3/users/glance` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.698214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05cc5c-cb45-4537-9644-15b4858df44c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:20.701849 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05cc5c-cb45-4537-9644-15b4858df44c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-6a05cc5c-cb45-4537-9644-15b4858df44c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:20.725754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:20.727740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:20.728739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:20.728739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:20.728739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:20.728739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:20.728739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:20.728739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:20.728739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:20.728739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:20.728739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:35:20.728739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:20.728739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 284/563] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:20.731838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05cc5c-cb45-4537-9644-15b4858df44c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:20.829177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e60d9-4ef5-4312-9867-472635e07147 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.852758 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:20.878457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40629545-142c-44d1-b6a2-dfa0b2a43b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.878962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40629545-142c-44d1-b6a2-dfa0b2a43b23 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.879270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40629545-142c-44d1-b6a2-dfa0b2a43b23 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.879529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40629545-142c-44d1-b6a2-dfa0b2a43b23 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.880203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40629545-142c-44d1-b6a2-dfa0b2a43b23 None admin] RBAC: Adding query filter params (name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:20.880493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40629545-142c-44d1-b6a2-dfa0b2a43b23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:20.881920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40629545-142c-44d1-b6a2-dfa0b2a43b23 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:20.896745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 280/564] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:20.899355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d41ce-1f8b-4392-8dc6-7bd54c33ed2b None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.899833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d41ce-1f8b-4392-8dc6-7bd54c33ed2b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.900301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d41ce-1f8b-4392-8dc6-7bd54c33ed2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.924924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05cc5c-cb45-4537-9644-15b4858df44c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.930845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-be9d41ce-1f8b-4392-8dc6-7bd54c33ed2b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:20.933574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-be9d41ce-1f8b-4392-8dc6-7bd54c33ed2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:20.966821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc40f4-82c4-4781-9c3b-472118ae597a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.967817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc40f4-82c4-4781-9c3b-472118ae597a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.968269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc40f4-82c4-4781-9c3b-472118ae597a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.968716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc40f4-82c4-4781-9c3b-472118ae597a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.987396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc40f4-82c4-4781-9c3b-472118ae597a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:20.989910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc40f4-82c4-4781-9c3b-472118ae597a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:20.992676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-a4cc40f4-82c4-4781-9c3b-472118ae597a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:21.018831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:21.020288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:21.021158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:21.021158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:21.021158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:21.021158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 285/565] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:21.022259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc40f4-82c4-4781-9c3b-472118ae597a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.035374 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 281/566] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:21.036232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d41ce-1f8b-4392-8dc6-7bd54c33ed2b None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.171817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc40f4-82c4-4781-9c3b-472118ae597a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.194735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d41ce-1f8b-4392-8dc6-7bd54c33ed2b None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.204731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:21.220421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5ccfb-48fd-479e-abe7-045fd94f86ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.221918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5ccfb-48fd-479e-abe7-045fd94f86ca None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.222383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5ccfb-48fd-479e-abe7-045fd94f86ca None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.222991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5ccfb-48fd-479e-abe7-045fd94f86ca None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.224427 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5ccfb-48fd-479e-abe7-045fd94f86ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:21.224935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5ccfb-48fd-479e-abe7-045fd94f86ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:21.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5ccfb-48fd-479e-abe7-045fd94f86ca None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:21.256557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0209a60-1f4e-4465-b087-7c80ddbac0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.257911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0209a60-1f4e-4465-b087-7c80ddbac0b1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.258568 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0209a60-1f4e-4465-b087-7c80ddbac0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.258726 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 286/567] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:21.259373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0209a60-1f4e-4465-b087-7c80ddbac0b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.260232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5ccfb-48fd-479e-abe7-045fd94f86ca None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.281263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0209a60-1f4e-4465-b087-7c80ddbac0b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:21.283257 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0209a60-1f4e-4465-b087-7c80ddbac0b1 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-c0209a60-1f4e-4465-b087-7c80ddbac0b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:21.302108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:21.304989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:21.306697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:21.306697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:21.306697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:21.306697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 282/568] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:21.306697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0209a60-1f4e-4465-b087-7c80ddbac0b1 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.342533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:21.407299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5ccfb-48fd-479e-abe7-045fd94f86ca None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.434809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0209a60-1f4e-4465-b087-7c80ddbac0b1 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.454983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db82c4-223a-4651-8892-886a01d4dcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.455861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db82c4-223a-4651-8892-886a01d4dcc4 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.456229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db82c4-223a-4651-8892-886a01d4dcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.456504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db82c4-223a-4651-8892-886a01d4dcc4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.469846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db82c4-223a-4651-8892-886a01d4dcc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:21.474205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794069d6-ab27-414f-9bd7-576abbb97605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-49db82c4-223a-4651-8892-886a01d4dcc4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:21.474817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:21.476757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:21.478617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:21.478617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:21.478617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:21.478617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:21.478617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:21.478617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:21.478617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:21.478617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:21.478617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:21.478617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:21.478617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:21.478617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:21.481453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794069d6-ab27-414f-9bd7-576abbb97605 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.481453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794069d6-ab27-414f-9bd7-576abbb97605 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.481453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794069d6-ab27-414f-9bd7-576abbb97605 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.481453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794069d6-ab27-414f-9bd7-576abbb97605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:21.481453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794069d6-ab27-414f-9bd7-576abbb97605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:21.482032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 287/569] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:21.482032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db82c4-223a-4651-8892-886a01d4dcc4 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.482283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794069d6-ab27-414f-9bd7-576abbb97605 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:21.500043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 283/570] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:21.500043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794069d6-ab27-414f-9bd7-576abbb97605 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.525076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:21.613393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db82c4-223a-4651-8892-886a01d4dcc4 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.635881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:21.641320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794069d6-ab27-414f-9bd7-576abbb97605 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.662277 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14636bdf-b0bd-48e7-89a8-0f8fecb21f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.663070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14636bdf-b0bd-48e7-89a8-0f8fecb21f13 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.663465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14636bdf-b0bd-48e7-89a8-0f8fecb21f13 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.663834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14636bdf-b0bd-48e7-89a8-0f8fecb21f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.676774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14636bdf-b0bd-48e7-89a8-0f8fecb21f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:21.678487 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14636bdf-b0bd-48e7-89a8-0f8fecb21f13 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:21.687922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9a6aa-ecae-4f88-a007-767a449e3d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.688663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9a6aa-ecae-4f88-a007-767a449e3d32 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.689037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9a6aa-ecae-4f88-a007-767a449e3d32 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.689407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9a6aa-ecae-4f88-a007-767a449e3d32 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.690335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9a6aa-ecae-4f88-a007-767a449e3d32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:21.690736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9a6aa-ecae-4f88-a007-767a449e3d32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:21.692397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9a6aa-ecae-4f88-a007-767a449e3d32 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-14636bdf-b0bd-48e7-89a8-0f8fecb21f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:21.697469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:21.699699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:21.701402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:21.701402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:21.701402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:21.701402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 288/571] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:21.701402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14636bdf-b0bd-48e7-89a8-0f8fecb21f13 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.709475 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 284/572] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:21.710662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9a6aa-ecae-4f88-a007-767a449e3d32 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.808989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:21.833408 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14636bdf-b0bd-48e7-89a8-0f8fecb21f13 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.841853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9a6aa-ecae-4f88-a007-767a449e3d32 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.876131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51c09d-891f-48cd-8e1c-c41ff90f00c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.876891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c09d-891f-48cd-8e1c-c41ff90f00c7 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.877319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c09d-891f-48cd-8e1c-c41ff90f00c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.877730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c09d-891f-48cd-8e1c-c41ff90f00c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.882202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fa46a-cfa6-4b18-bb5a-973db91515e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.882806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fa46a-cfa6-4b18-bb5a-973db91515e6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.883074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fa46a-cfa6-4b18-bb5a-973db91515e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.883369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fa46a-cfa6-4b18-bb5a-973db91515e6 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.884036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fa46a-cfa6-4b18-bb5a-973db91515e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:21.884330 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fa46a-cfa6-4b18-bb5a-973db91515e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:21.886460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fa46a-cfa6-4b18-bb5a-973db91515e6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:21.893717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f51c09d-891f-48cd-8e1c-c41ff90f00c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:21.905600 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 285/573] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:21.905600 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fa46a-cfa6-4b18-bb5a-973db91515e6 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-1f51c09d-891f-48cd-8e1c-c41ff90f00c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:21.905959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:21.907958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 289/574] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:21.909591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51c09d-891f-48cd-8e1c-c41ff90f00c7 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.946988 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:22.043675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fa46a-cfa6-4b18-bb5a-973db91515e6 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.046366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51c09d-891f-48cd-8e1c-c41ff90f00c7 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.093513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da45d7-1856-4a4e-91ef-afd7f6ea9872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.094379 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da45d7-1856-4a4e-91ef-afd7f6ea9872 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.094781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da45d7-1856-4a4e-91ef-afd7f6ea9872 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.095247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da45d7-1856-4a4e-91ef-afd7f6ea9872 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.096143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da45d7-1856-4a4e-91ef-afd7f6ea9872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:22.096585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da45d7-1856-4a4e-91ef-afd7f6ea9872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.098582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da45d7-1856-4a4e-91ef-afd7f6ea9872 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.098879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c70f27-ebb8-4ed7-9d54-2afafed9608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.099661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c70f27-ebb8-4ed7-9d54-2afafed9608a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.100030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c70f27-ebb8-4ed7-9d54-2afafed9608a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.100424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c70f27-ebb8-4ed7-9d54-2afafed9608a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.109453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c70f27-ebb8-4ed7-9d54-2afafed9608a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.111059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c70f27-ebb8-4ed7-9d54-2afafed9608a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.113297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:22.113842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 290/575] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:22.114971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da45d7-1856-4a4e-91ef-afd7f6ea9872 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-26c70f27-ebb8-4ed7-9d54-2afafed9608a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:22.137138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:22.139021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:22.141078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:22.141078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:22.141078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:22.141078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:22.141078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:22.141078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:22.141078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:22.141078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:22.141078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 286/576] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:22.141078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c70f27-ebb8-4ed7-9d54-2afafed9608a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.234980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:22.254179 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da45d7-1856-4a4e-91ef-afd7f6ea9872 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.267179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c70f27-ebb8-4ed7-9d54-2afafed9608a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.301449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc31de-2f0a-4c43-90c2-ffbd62d0badf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.302104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc31de-2f0a-4c43-90c2-ffbd62d0badf None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.302376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc31de-2f0a-4c43-90c2-ffbd62d0badf None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.302657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc31de-2f0a-4c43-90c2-ffbd62d0badf None admin] PATH_INFO: `/v3/users/nova` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.309294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd6d51-a1ea-4b28-91bb-6eb259bdef5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.310052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd6d51-a1ea-4b28-91bb-6eb259bdef5d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.310434 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd6d51-a1ea-4b28-91bb-6eb259bdef5d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.310876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd6d51-a1ea-4b28-91bb-6eb259bdef5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.311573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc31de-2f0a-4c43-90c2-ffbd62d0badf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.313092 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc31de-2f0a-4c43-90c2-ffbd62d0badf None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.325042 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd6d51-a1ea-4b28-91bb-6eb259bdef5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.326503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd6d51-a1ea-4b28-91bb-6eb259bdef5d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-b8fc31de-2f0a-4c43-90c2-ffbd62d0badf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:22.327614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:22.329000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:22.330740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:22.330740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:22.330740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:22.330740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:22.330740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:22.330740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:22.330740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:22.330740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:22.330740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:22.330740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:22.330740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 291/577] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:22.332599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc31de-2f0a-4c43-90c2-ffbd62d0badf None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-c5fd6d51-a1ea-4b28-91bb-6eb259bdef5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:22.342721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:22.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:22.348997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:22.348997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:22.348997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:22.348997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 287/578] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:22.348997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd6d51-a1ea-4b28-91bb-6eb259bdef5d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.432829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:22.453715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc31de-2f0a-4c43-90c2-ffbd62d0badf None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.492978 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcc44c-54ce-4063-a1f5-03950763b5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.493575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcc44c-54ce-4063-a1f5-03950763b5d8 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.493874 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcc44c-54ce-4063-a1f5-03950763b5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.494162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcc44c-54ce-4063-a1f5-03950763b5d8 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.495448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcc44c-54ce-4063-a1f5-03950763b5d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:22.495760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcc44c-54ce-4063-a1f5-03950763b5d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.497290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcc44c-54ce-4063-a1f5-03950763b5d8 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.505956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 292/579] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:22.506811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcc44c-54ce-4063-a1f5-03950763b5d8 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.519738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:22.525101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd6d51-a1ea-4b28-91bb-6eb259bdef5d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.578338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185bab9-a000-4ea5-ad3a-39b3c65aface None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.579053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185bab9-a000-4ea5-ad3a-39b3c65aface None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.579391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185bab9-a000-4ea5-ad3a-39b3c65aface None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.579770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185bab9-a000-4ea5-ad3a-39b3c65aface None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.587884 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185bab9-a000-4ea5-ad3a-39b3c65aface None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.590115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185bab9-a000-4ea5-ad3a-39b3c65aface None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-b185bab9-a000-4ea5-ad3a-39b3c65aface None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:22.602800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:22.612736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:22.614741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:22.614741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:22.614741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:22.614741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:22.614741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:22.614741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:22.614741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:22.614741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:22.614741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 288/580] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:22.614741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185bab9-a000-4ea5-ad3a-39b3c65aface None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.678517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcc44c-54ce-4063-a1f5-03950763b5d8 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.719313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185bab9-a000-4ea5-ad3a-39b3c65aface None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.729874 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdbb8b-cfca-416d-826d-ef97ba54b441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.730601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdbb8b-cfca-416d-826d-ef97ba54b441 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.730890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdbb8b-cfca-416d-826d-ef97ba54b441 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.731190 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdbb8b-cfca-416d-826d-ef97ba54b441 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.731847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:22.732004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdbb8b-cfca-416d-826d-ef97ba54b441 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:22.732201 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdbb8b-cfca-416d-826d-ef97ba54b441 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.733821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdbb8b-cfca-416d-826d-ef97ba54b441 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.749901 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 293/581] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:22.750900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdbb8b-cfca-416d-826d-ef97ba54b441 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.765785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b9a0f-7f56-4a4c-bfc7-d3a20da60058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.766437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b9a0f-7f56-4a4c-bfc7-d3a20da60058 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.766756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b9a0f-7f56-4a4c-bfc7-d3a20da60058 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.767145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b9a0f-7f56-4a4c-bfc7-d3a20da60058 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.768010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b9a0f-7f56-4a4c-bfc7-d3a20da60058 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:22.768503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b9a0f-7f56-4a4c-bfc7-d3a20da60058 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.770147 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b9a0f-7f56-4a4c-bfc7-d3a20da60058 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.793165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 289/582] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:22.794560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b9a0f-7f56-4a4c-bfc7-d3a20da60058 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.835628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:22.886547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdbb8b-cfca-416d-826d-ef97ba54b441 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.913376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b9a0f-7f56-4a4c-bfc7-d3a20da60058 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.942045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579dae76-00e3-40b5-adb0-eb363bd1312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.942663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dae76-00e3-40b5-adb0-eb363bd1312f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.942941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dae76-00e3-40b5-adb0-eb363bd1312f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.943248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dae76-00e3-40b5-adb0-eb363bd1312f None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.943908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579dae76-00e3-40b5-adb0-eb363bd1312f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:22.944207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579dae76-00e3-40b5-adb0-eb363bd1312f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.945620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579dae76-00e3-40b5-adb0-eb363bd1312f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.959694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 294/583] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:22.960656 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579dae76-00e3-40b5-adb0-eb363bd1312f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.961790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eecade-7f95-40cd-a362-31ee065e44a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.962631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eecade-7f95-40cd-a362-31ee065e44a7 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.963799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eecade-7f95-40cd-a362-31ee065e44a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.964386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eecade-7f95-40cd-a362-31ee065e44a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.983943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eecade-7f95-40cd-a362-31ee065e44a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.986683 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eecade-7f95-40cd-a362-31ee065e44a7 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-e0eecade-7f95-40cd-a362-31ee065e44a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:23.014494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:23.015979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:23.017059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:23.017059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:23.017059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:23.017059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 290/584] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:23.017059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eecade-7f95-40cd-a362-31ee065e44a7 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.043525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:23.098356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579dae76-00e3-40b5-adb0-eb363bd1312f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.126526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:23.137663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b26425-828d-4a9b-90cb-21c951f41334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.138366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b26425-828d-4a9b-90cb-21c951f41334 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.138646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b26425-828d-4a9b-90cb-21c951f41334 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.138924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b26425-828d-4a9b-90cb-21c951f41334 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.139825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b26425-828d-4a9b-90cb-21c951f41334 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:23.140453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b26425-828d-4a9b-90cb-21c951f41334 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.142115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b26425-828d-4a9b-90cb-21c951f41334 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.155872 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eecade-7f95-40cd-a362-31ee065e44a7 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.157364 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 295/585] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:23.158228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b26425-828d-4a9b-90cb-21c951f41334 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.200035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825a9fb-ab14-44c7-970b-ca0dabbe7e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.201062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825a9fb-ab14-44c7-970b-ca0dabbe7e28 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.201543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825a9fb-ab14-44c7-970b-ca0dabbe7e28 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.202022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825a9fb-ab14-44c7-970b-ca0dabbe7e28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.203229 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825a9fb-ab14-44c7-970b-ca0dabbe7e28 None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=c5cf86ae159749cea0cc9783bffbfe75) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:23.203791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825a9fb-ab14-44c7-970b-ca0dabbe7e28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.206389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825a9fb-ab14-44c7-970b-ca0dabbe7e28 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.217677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 291/586] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/role_assignments?user.id=c5cf86ae159749cea0cc9783bffbfe75&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:23.218899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825a9fb-ab14-44c7-970b-ca0dabbe7e28 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.313777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b26425-828d-4a9b-90cb-21c951f41334 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.364061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc17a5b-f111-4843-b69c-cc72abb0cc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.365425 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc17a5b-f111-4843-b69c-cc72abb0cc4e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.365908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc17a5b-f111-4843-b69c-cc72abb0cc4e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.366403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc17a5b-f111-4843-b69c-cc72abb0cc4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.380297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:23.387639 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc17a5b-f111-4843-b69c-cc72abb0cc4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.389709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc17a5b-f111-4843-b69c-cc72abb0cc4e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.404940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825a9fb-ab14-44c7-970b-ca0dabbe7e28 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-9fc17a5b-f111-4843-b69c-cc72abb0cc4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:23.408048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:23.409000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:23.409879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:23.409879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:23.409879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:23.413985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 296/587] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:23.415834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc17a5b-f111-4843-b69c-cc72abb0cc4e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.458688 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39582f34-b0dc-4a95-82a0-843ccc15bffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.460626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39582f34-b0dc-4a95-82a0-843ccc15bffe None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.460626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39582f34-b0dc-4a95-82a0-843ccc15bffe None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.460833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39582f34-b0dc-4a95-82a0-843ccc15bffe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.464548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:23.477659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39582f34-b0dc-4a95-82a0-843ccc15bffe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.480062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39582f34-b0dc-4a95-82a0-843ccc15bffe None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-39582f34-b0dc-4a95-82a0-843ccc15bffe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:23.501049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:23.502925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:23.506091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:23.506091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:23.506091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:23.506091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:23.506091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:23.506091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:23.506091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:23.506091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:23.506091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:35:23.506091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:23.506091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 292/588] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:23.506091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39582f34-b0dc-4a95-82a0-843ccc15bffe None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.574233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc17a5b-f111-4843-b69c-cc72abb0cc4e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.633854 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1ea81-1153-4e23-b97e-6b3148c93ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.635799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1ea81-1153-4e23-b97e-6b3148c93ef3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.635983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1ea81-1153-4e23-b97e-6b3148c93ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.636471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1ea81-1153-4e23-b97e-6b3148c93ef3 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.637688 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1ea81-1153-4e23-b97e-6b3148c93ef3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:23.638222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1ea81-1153-4e23-b97e-6b3148c93ef3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.640837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1ea81-1153-4e23-b97e-6b3148c93ef3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.662664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 297/589] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:23.663703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39582f34-b0dc-4a95-82a0-843ccc15bffe None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.664263 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1ea81-1153-4e23-b97e-6b3148c93ef3 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.723346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840ca63-24a6-412d-b268-5a33a31bec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.724026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840ca63-24a6-412d-b268-5a33a31bec84 None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.724305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840ca63-24a6-412d-b268-5a33a31bec84 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.724567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840ca63-24a6-412d-b268-5a33a31bec84 None admin] PATH_INFO: `/v3/projects/8ef8b586382941408cd88ee654c42d32/users/06fad9466df14827983dac2b4b844f3a/roles/dcba9d07b31a4387b3014365e6e2ea10` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.741750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840ca63-24a6-412d-b268-5a33a31bec84 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef8b586382941408cd88ee654c42d32, user_id=06fad9466df14827983dac2b4b844f3a, role_id=dcba9d07b31a4387b3014365e6e2ea10)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.743238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840ca63-24a6-412d-b268-5a33a31bec84 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.745838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:23.762589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 293/590] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:23 2022] PUT /identity/v3/projects/8ef8b586382941408cd88ee654c42d32/users/06fad9466df14827983dac2b4b844f3a/roles/dcba9d07b31a4387b3014365e6e2ea10 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:35:23.763415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840ca63-24a6-412d-b268-5a33a31bec84 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.824531 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:23.831685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1ea81-1153-4e23-b97e-6b3148c93ef3 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.889437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dbf1f-6320-4c4b-a85b-8079a243222a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.890775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dbf1f-6320-4c4b-a85b-8079a243222a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.891146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dbf1f-6320-4c4b-a85b-8079a243222a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.891503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dbf1f-6320-4c4b-a85b-8079a243222a None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.893027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dbf1f-6320-4c4b-a85b-8079a243222a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:23.893409 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dbf1f-6320-4c4b-a85b-8079a243222a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.894987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dbf1f-6320-4c4b-a85b-8079a243222a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.914232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840ca63-24a6-412d-b268-5a33a31bec84 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.918640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 298/591] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:23.920438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dbf1f-6320-4c4b-a85b-8079a243222a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.959988 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835b091-5507-48c7-b41d-7da5e59aa278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.960951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835b091-5507-48c7-b41d-7da5e59aa278 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.961386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835b091-5507-48c7-b41d-7da5e59aa278 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.961806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835b091-5507-48c7-b41d-7da5e59aa278 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.962744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835b091-5507-48c7-b41d-7da5e59aa278 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:23.963836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835b091-5507-48c7-b41d-7da5e59aa278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.965870 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835b091-5507-48c7-b41d-7da5e59aa278 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.990226 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 294/592] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:23.991011 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835b091-5507-48c7-b41d-7da5e59aa278 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.078353 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:24.141527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dbf1f-6320-4c4b-a85b-8079a243222a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.150023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835b091-5507-48c7-b41d-7da5e59aa278 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.199969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97abdb2-6b66-4e6b-84f9-1eda9326e98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.201138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97abdb2-6b66-4e6b-84f9-1eda9326e98a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.201976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97abdb2-6b66-4e6b-84f9-1eda9326e98a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.202526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97abdb2-6b66-4e6b-84f9-1eda9326e98a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.211557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97abdb2-6b66-4e6b-84f9-1eda9326e98a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:24.213543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97abdb2-6b66-4e6b-84f9-1eda9326e98a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:24.215037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88a5b6-adfc-40b8-ae8f-7863675ed93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.216419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a5b6-adfc-40b8-ae8f-7863675ed93a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.216921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a5b6-adfc-40b8-ae8f-7863675ed93a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.217409 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a5b6-adfc-40b8-ae8f-7863675ed93a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-c97abdb2-6b66-4e6b-84f9-1eda9326e98a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:24.229352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:24.230300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:24.231185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:24.231185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:24.231185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:24.231185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:24.231185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:24.231185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:24.231185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:24.231185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:24.234009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88a5b6-adfc-40b8-ae8f-7863675ed93a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:24.234309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 295/593] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:24.235436 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97abdb2-6b66-4e6b-84f9-1eda9326e98a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.236792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88a5b6-adfc-40b8-ae8f-7863675ed93a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-2f88a5b6-adfc-40b8-ae8f-7863675ed93a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:24.255452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:24.256421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:24.257338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:24.257338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:24.257338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:24.257338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:24.257338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:24.257338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:24.257338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:24.257338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:24.257338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:35:24.257338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:24.264623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 299/594] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:24.266166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88a5b6-adfc-40b8-ae8f-7863675ed93a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.287618 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:24.362906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97abdb2-6b66-4e6b-84f9-1eda9326e98a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.380720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:24.383383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88a5b6-adfc-40b8-ae8f-7863675ed93a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.414537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedba67-d30e-4938-ab63-a37c2f68dd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.415437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedba67-d30e-4938-ab63-a37c2f68dd5e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.415813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedba67-d30e-4938-ab63-a37c2f68dd5e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.416777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedba67-d30e-4938-ab63-a37c2f68dd5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.431790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedba67-d30e-4938-ab63-a37c2f68dd5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:24.434072 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec48ca-8429-4689-85c2-6d310347dbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.434306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedba67-d30e-4938-ab63-a37c2f68dd5e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:24.435662 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec48ca-8429-4689-85c2-6d310347dbd3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.436247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec48ca-8429-4689-85c2-6d310347dbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.436829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec48ca-8429-4689-85c2-6d310347dbd3 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.437945 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec48ca-8429-4689-85c2-6d310347dbd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:24.438572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec48ca-8429-4689-85c2-6d310347dbd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:24.440851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec48ca-8429-4689-85c2-6d310347dbd3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-ccedba67-d30e-4938-ab63-a37c2f68dd5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:24.451652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:24.452838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:24.453913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:24.453913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:24.453913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:24.455479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 296/595] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:24.456515 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedba67-d30e-4938-ab63-a37c2f68dd5e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.466975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 300/596] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:24.468754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec48ca-8429-4689-85c2-6d310347dbd3 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.605291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:24.616567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedba67-d30e-4938-ab63-a37c2f68dd5e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.646509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec48ca-8429-4689-85c2-6d310347dbd3 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.669910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b23f5-3a7b-4cec-8230-a95d1b1ec0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.671161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b23f5-3a7b-4cec-8230-a95d1b1ec0bd None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.671753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b23f5-3a7b-4cec-8230-a95d1b1ec0bd None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.672458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b23f5-3a7b-4cec-8230-a95d1b1ec0bd None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.674008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b23f5-3a7b-4cec-8230-a95d1b1ec0bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:24.675259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b23f5-3a7b-4cec-8230-a95d1b1ec0bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:24.677624 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b23f5-3a7b-4cec-8230-a95d1b1ec0bd None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:24.705771 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 297/597] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Aug 31 09:35:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:24.707065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b23f5-3a7b-4cec-8230-a95d1b1ec0bd None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.719770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfa24d-bb24-4b02-9923-14d8397068a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.721048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfa24d-bb24-4b02-9923-14d8397068a0 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.721734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfa24d-bb24-4b02-9923-14d8397068a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.722340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfa24d-bb24-4b02-9923-14d8397068a0 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.723860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfa24d-bb24-4b02-9923-14d8397068a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:24.724344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfa24d-bb24-4b02-9923-14d8397068a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:24.726904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfa24d-bb24-4b02-9923-14d8397068a0 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:24.746763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:24.750300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 301/598] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:24.751543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfa24d-bb24-4b02-9923-14d8397068a0 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.865233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b23f5-3a7b-4cec-8230-a95d1b1ec0bd None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.918087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda25f53-4f9a-4f09-bddd-5a7272aa0532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.919625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25f53-4f9a-4f09-bddd-5a7272aa0532 None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.920083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25f53-4f9a-4f09-bddd-5a7272aa0532 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.920642 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25f53-4f9a-4f09-bddd-5a7272aa0532 None admin] PATH_INFO: `/v3/projects/8ef8b586382941408cd88ee654c42d32/users/ef72bb95992c4a21ae2dbb2a08c51421/roles/dcba9d07b31a4387b3014365e6e2ea10` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.928969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfa24d-bb24-4b02-9923-14d8397068a0 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.946545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda25f53-4f9a-4f09-bddd-5a7272aa0532 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef8b586382941408cd88ee654c42d32, user_id=ef72bb95992c4a21ae2dbb2a08c51421, role_id=dcba9d07b31a4387b3014365e6e2ea10)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:24.950052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda25f53-4f9a-4f09-bddd-5a7272aa0532 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:24.974071 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 298/599] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:24 2022] PUT /identity/v3/projects/8ef8b586382941408cd88ee654c42d32/users/ef72bb95992c4a21ae2dbb2a08c51421/roles/dcba9d07b31a4387b3014365e6e2ea10 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:35:24.975361 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda25f53-4f9a-4f09-bddd-5a7272aa0532 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.983243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be46c21-5b75-4365-a019-689b9063b6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.984326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be46c21-5b75-4365-a019-689b9063b6eb None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.984849 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be46c21-5b75-4365-a019-689b9063b6eb None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.985398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be46c21-5b75-4365-a019-689b9063b6eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.997119 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:25.002384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be46c21-5b75-4365-a019-689b9063b6eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:25.004096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be46c21-5b75-4365-a019-689b9063b6eb None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-7be46c21-5b75-4365-a019-689b9063b6eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:25.021949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:25.023164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:25.024276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:25.024276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:25.024276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:25.030239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 302/600] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:25.031737 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be46c21-5b75-4365-a019-689b9063b6eb None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:25.138465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:25.212588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda25f53-4f9a-4f09-bddd-5a7272aa0532 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:25.274051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be46c21-5b75-4365-a019-689b9063b6eb None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:25.274283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b5d80-9c3c-4932-a5d0-e0cd31a95e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:25.274845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b5d80-9c3c-4932-a5d0-e0cd31a95e9d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:25.275156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b5d80-9c3c-4932-a5d0-e0cd31a95e9d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:25.275419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b5d80-9c3c-4932-a5d0-e0cd31a95e9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:25.285979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b5d80-9c3c-4932-a5d0-e0cd31a95e9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:25.288196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b5d80-9c3c-4932-a5d0-e0cd31a95e9d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-9d4b5d80-9c3c-4932-a5d0-e0cd31a95e9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:25.304232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:25.305247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:25.306151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:25.306151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:25.306151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:25.306151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:25.306151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:25.306151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:25.306151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:25.306151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:25.306151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 299/601] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:25.310118 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394455c7-09a4-4dfa-ae8d-851286287766 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:25.310376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394455c7-09a4-4dfa-ae8d-851286287766 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:25.310604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394455c7-09a4-4dfa-ae8d-851286287766 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:25.312364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 300/602] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:25.315958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394455c7-09a4-4dfa-ae8d-851286287766 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:25.322298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6bebd-e2ea-4b97-a27c-228ccb730ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:25.323838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6bebd-e2ea-4b97-a27c-228ccb730ca6 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:25.324424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6bebd-e2ea-4b97-a27c-228ccb730ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:25.324932 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6bebd-e2ea-4b97-a27c-228ccb730ca6 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:25.326421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6bebd-e2ea-4b97-a27c-228ccb730ca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:25.326975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6bebd-e2ea-4b97-a27c-228ccb730ca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:25.328485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6bebd-e2ea-4b97-a27c-228ccb730ca6 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:25.344706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 303/603] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:25.345998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19be7c-d45f-4608-8f6b-f8d46f191a8e None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:25.346310 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19be7c-d45f-4608-8f6b-f8d46f191a8e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:25.346581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19be7c-d45f-4608-8f6b-f8d46f191a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:25.372028 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-6f19be7c-d45f-4608-8f6b-f8d46f191a8e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:25.375428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6f19be7c-d45f-4608-8f6b-f8d46f191a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:25.440832 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:25.453617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394455c7-09a4-4dfa-ae8d-851286287766 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:25.476784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 304/604] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:25.483491 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19be7c-d45f-4608-8f6b-f8d46f191a8e None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:25.515619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:25.525553 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29492f-cfaa-442d-8df4-b3d4731cb475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:25.526176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29492f-cfaa-442d-8df4-b3d4731cb475 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:25.526452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29492f-cfaa-442d-8df4-b3d4731cb475 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:25.526707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29492f-cfaa-442d-8df4-b3d4731cb475 None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:25.527442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29492f-cfaa-442d-8df4-b3d4731cb475 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:25.527744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29492f-cfaa-442d-8df4-b3d4731cb475 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:25.534541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29492f-cfaa-442d-8df4-b3d4731cb475 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:25.565340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 301/605] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:25.569396 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29492f-cfaa-442d-8df4-b3d4731cb475 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:25.713720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29492f-cfaa-442d-8df4-b3d4731cb475 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:25.753268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd7e97-ea28-4922-a24d-fd6dd180c90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:25.753845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd7e97-ea28-4922-a24d-fd6dd180c90e None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:25.754119 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd7e97-ea28-4922-a24d-fd6dd180c90e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:25.754444 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd7e97-ea28-4922-a24d-fd6dd180c90e None admin] PATH_INFO: `/v3/projects/8ef8b586382941408cd88ee654c42d32/users/6ff1177119cb48e0a88b0f11a6b1b0c3/roles/dcba9d07b31a4387b3014365e6e2ea10` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:25.757340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19be7c-d45f-4608-8f6b-f8d46f191a8e None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:25.772622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd7e97-ea28-4922-a24d-fd6dd180c90e None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef8b586382941408cd88ee654c42d32, user_id=6ff1177119cb48e0a88b0f11a6b1b0c3, role_id=dcba9d07b31a4387b3014365e6e2ea10)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:25.774064 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd7e97-ea28-4922-a24d-fd6dd180c90e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:25.796456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 302/606] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:25 2022] PUT /identity/v3/projects/8ef8b586382941408cd88ee654c42d32/users/6ff1177119cb48e0a88b0f11a6b1b0c3/roles/dcba9d07b31a4387b3014365e6e2ea10 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:35:25.797413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd7e97-ea28-4922-a24d-fd6dd180c90e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:25.864816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0bbe9-5dc6-44e8-a44a-571fb5da1503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:25.866065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0bbe9-5dc6-44e8-a44a-571fb5da1503 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:25.866560 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0bbe9-5dc6-44e8-a44a-571fb5da1503 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:25.867048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0bbe9-5dc6-44e8-a44a-571fb5da1503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:25.891521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0bbe9-5dc6-44e8-a44a-571fb5da1503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:25.898963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0bbe9-5dc6-44e8-a44a-571fb5da1503 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-7fb0bbe9-5dc6-44e8-a44a-571fb5da1503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:25.943161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:25.944128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:25.945000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:25.945000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:25.945000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:25.945000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:25.945000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:25.945000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:25.945000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:25.945000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:25.945000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 305/607] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 465 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:25.952719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0bbe9-5dc6-44e8-a44a-571fb5da1503 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:25.961993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:26.016093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:26.033353 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd7e97-ea28-4922-a24d-fd6dd180c90e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:26.085203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98ebfe-ec2c-4b89-acdc-fadc391d05db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:26.085949 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98ebfe-ec2c-4b89-acdc-fadc391d05db None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.086226 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98ebfe-ec2c-4b89-acdc-fadc391d05db None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.086494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98ebfe-ec2c-4b89-acdc-fadc391d05db None admin] PATH_INFO: `/v3/roles/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.101151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98ebfe-ec2c-4b89-acdc-fadc391d05db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:26.106112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0bbe9-5dc6-44e8-a44a-571fb5da1503 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-4f98ebfe-ec2c-4b89-acdc-fadc391d05db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:26.110655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:26.111801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:26.112899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 303/608] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:26.124194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98ebfe-ec2c-4b89-acdc-fadc391d05db None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:26.148108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d4cde-84ee-4160-a345-8fc51efb2620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:26.149754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d4cde-84ee-4160-a345-8fc51efb2620 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.150177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d4cde-84ee-4160-a345-8fc51efb2620 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.151350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d4cde-84ee-4160-a345-8fc51efb2620 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.152672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d4cde-84ee-4160-a345-8fc51efb2620 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:26.153128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d4cde-84ee-4160-a345-8fc51efb2620 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:26.154779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d4cde-84ee-4160-a345-8fc51efb2620 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:26.170540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 306/609] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:26.180968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d4cde-84ee-4160-a345-8fc51efb2620 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:26.321913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98ebfe-ec2c-4b89-acdc-fadc391d05db None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:26.324100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:26.364137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae85d1-63ac-447b-9532-c1d779ebe121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:26.365164 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae85d1-63ac-447b-9532-c1d779ebe121 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.365643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae85d1-63ac-447b-9532-c1d779ebe121 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.366104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae85d1-63ac-447b-9532-c1d779ebe121 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.366984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae85d1-63ac-447b-9532-c1d779ebe121 None admin] RBAC: Adding query filter params (name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:26.367518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae85d1-63ac-447b-9532-c1d779ebe121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:26.367624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:26.369578 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae85d1-63ac-447b-9532-c1d779ebe121 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:26.374760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d4cde-84ee-4160-a345-8fc51efb2620 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:26.380419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 304/610] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:26.390904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae85d1-63ac-447b-9532-c1d779ebe121 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:26.439959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10c166-b529-4824-a22e-621a35a1d5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:26.441105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10c166-b529-4824-a22e-621a35a1d5e2 None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.441577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10c166-b529-4824-a22e-621a35a1d5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.442081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10c166-b529-4824-a22e-621a35a1d5e2 None admin] PATH_INFO: `/v3/projects/8ef8b586382941408cd88ee654c42d32/users/21d8ade66b7e4a1880c961c5e13adf04/roles/dcba9d07b31a4387b3014365e6e2ea10` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.471864 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10c166-b529-4824-a22e-621a35a1d5e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef8b586382941408cd88ee654c42d32, user_id=21d8ade66b7e4a1880c961c5e13adf04, role_id=dcba9d07b31a4387b3014365e6e2ea10)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:26.474328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10c166-b529-4824-a22e-621a35a1d5e2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:26.507194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 307/611] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:26 2022] PUT /identity/v3/projects/8ef8b586382941408cd88ee654c42d32/users/21d8ade66b7e4a1880c961c5e13adf04/roles/dcba9d07b31a4387b3014365e6e2ea10 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:35:26.507194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed5692-153f-4cad-a52f-d1936581c723 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.507194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed5692-153f-4cad-a52f-d1936581c723 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.507194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed5692-153f-4cad-a52f-d1936581c723 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.508067 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 308/612] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:26.521742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2811b-9a04-4bd6-80ee-c755416ec133 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.522449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2811b-9a04-4bd6-80ee-c755416ec133 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.523262 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2811b-9a04-4bd6-80ee-c755416ec133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.535541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae85d1-63ac-447b-9532-c1d779ebe121 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:26.563421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ddb2811b-9a04-4bd6-80ee-c755416ec133 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:26.565941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ddb2811b-9a04-4bd6-80ee-c755416ec133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:26.629802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56e378-99fb-49d4-866d-0c08e856aaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:26.630801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56e378-99fb-49d4-866d-0c08e856aaf8 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.631313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56e378-99fb-49d4-866d-0c08e856aaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.631792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56e378-99fb-49d4-866d-0c08e856aaf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.676264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56e378-99fb-49d4-866d-0c08e856aaf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:26.679010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56e378-99fb-49d4-866d-0c08e856aaf8 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-0b56e378-99fb-49d4-866d-0c08e856aaf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:26.727712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:26.730074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:26.731269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:26.731269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:26.731269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:26.731269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 305/613] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:26.737909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56e378-99fb-49d4-866d-0c08e856aaf8 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:26.763495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 309/614] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:26.770564 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:26.883882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56e378-99fb-49d4-866d-0c08e856aaf8 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:26.960021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea69433-255c-46fe-a813-24f95fdf1fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:26.960941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea69433-255c-46fe-a813-24f95fdf1fe9 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.961364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea69433-255c-46fe-a813-24f95fdf1fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.961778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea69433-255c-46fe-a813-24f95fdf1fe9 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.962811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea69433-255c-46fe-a813-24f95fdf1fe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:26.964730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea69433-255c-46fe-a813-24f95fdf1fe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:26.967053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea69433-255c-46fe-a813-24f95fdf1fe9 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:26.991066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 306/615] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:26.997642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2811b-9a04-4bd6-80ee-c755416ec133 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.185583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:27.216943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2811b-9a04-4bd6-80ee-c755416ec133 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:27.218416 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea69433-255c-46fe-a813-24f95fdf1fe9 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.290355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fb4f5-21cd-42c0-b44b-37218db79d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:27.292337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb4f5-21cd-42c0-b44b-37218db79d35 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.293015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb4f5-21cd-42c0-b44b-37218db79d35 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.293447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb4f5-21cd-42c0-b44b-37218db79d35 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.316254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fb4f5-21cd-42c0-b44b-37218db79d35 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:27.319521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fb4f5-21cd-42c0-b44b-37218db79d35 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-640fb4f5-21cd-42c0-b44b-37218db79d35 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:27.341407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:27.342428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:27.343997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:27.343997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:27.343997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:27.343997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:27.343997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:27.343997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:27.343997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:27.343997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:27.343997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:27.343997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:27.343997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 310/616] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:27.347627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:27.352256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fb4f5-21cd-42c0-b44b-37218db79d35 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.375837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea69433-255c-46fe-a813-24f95fdf1fe9 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:27.415259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645ea1a-4366-4586-a392-a60fa6509e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:27.416085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645ea1a-4366-4586-a392-a60fa6509e0f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.416414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645ea1a-4366-4586-a392-a60fa6509e0f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.416679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645ea1a-4366-4586-a392-a60fa6509e0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.435157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645ea1a-4366-4586-a392-a60fa6509e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-c645ea1a-4366-4586-a392-a60fa6509e0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:27.438907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:27.439866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:27.440742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:27.440742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:27.440742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:27.440742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:27.440742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:27.440742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:27.440742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:27.440742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:27.440742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:27.440742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:27.440742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:27.440742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:27.442029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 307/617] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:27.449488 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645ea1a-4366-4586-a392-a60fa6509e0f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.586379 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fb4f5-21cd-42c0-b44b-37218db79d35 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:27.621668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645ea1a-4366-4586-a392-a60fa6509e0f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:27.688660 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f91b6-31c9-4749-91bc-4eb8ae939294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:27.689870 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f91b6-31c9-4749-91bc-4eb8ae939294 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.690370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f91b6-31c9-4749-91bc-4eb8ae939294 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.690934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f91b6-31c9-4749-91bc-4eb8ae939294 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.692439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f91b6-31c9-4749-91bc-4eb8ae939294 None admin] RBAC: Adding query filter params (name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:27.693065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f91b6-31c9-4749-91bc-4eb8ae939294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:27.695757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f91b6-31c9-4749-91bc-4eb8ae939294 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:27.706649 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287ce8-dfe3-425b-90a0-9e97a5e3013e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:27.707324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287ce8-dfe3-425b-90a0-9e97a5e3013e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.707611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287ce8-dfe3-425b-90a0-9e97a5e3013e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.707885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287ce8-dfe3-425b-90a0-9e97a5e3013e None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.708634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287ce8-dfe3-425b-90a0-9e97a5e3013e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:27.708943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287ce8-dfe3-425b-90a0-9e97a5e3013e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:27.710383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287ce8-dfe3-425b-90a0-9e97a5e3013e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:27.713041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 308/618] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:27.714205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:27.719059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f91b6-31c9-4749-91bc-4eb8ae939294 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.745203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:27.762230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 311/619] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:27.768055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287ce8-dfe3-425b-90a0-9e97a5e3013e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.863331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f91b6-31c9-4749-91bc-4eb8ae939294 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:27.931536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6891ce4-9464-4890-bf5a-448082ed809d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:27.933004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6891ce4-9464-4890-bf5a-448082ed809d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.933748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6891ce4-9464-4890-bf5a-448082ed809d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.934440 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6891ce4-9464-4890-bf5a-448082ed809d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.936089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287ce8-dfe3-425b-90a0-9e97a5e3013e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:27.956510 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6891ce4-9464-4890-bf5a-448082ed809d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:27.959639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6891ce4-9464-4890-bf5a-448082ed809d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-b6891ce4-9464-4890-bf5a-448082ed809d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:27.990040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:27.991992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:27.993705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:27.993705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:27.993705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:27.993705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 309/620] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:27.996404 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4011ac-98a2-4787-8c40-9a3105245420 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.996664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4011ac-98a2-4787-8c40-9a3105245420 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.997113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4011ac-98a2-4787-8c40-9a3105245420 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.999936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 310/621] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:28.001120 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4011ac-98a2-4787-8c40-9a3105245420 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.002503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebac4e4-0664-4ac2-8061-daba38995244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.003413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebac4e4-0664-4ac2-8061-daba38995244 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.003913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebac4e4-0664-4ac2-8061-daba38995244 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.004405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebac4e4-0664-4ac2-8061-daba38995244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.018100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebac4e4-0664-4ac2-8061-daba38995244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.020579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebac4e4-0664-4ac2-8061-daba38995244 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-4ebac4e4-0664-4ac2-8061-daba38995244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:28.041693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:28.042805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:28.043878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:28.043878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:28.043878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:28.045452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 312/622] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:28.046643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0755b86-12f9-4146-95bf-0f85b2ec48ae None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.047021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0755b86-12f9-4146-95bf-0f85b2ec48ae None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.047880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0755b86-12f9-4146-95bf-0f85b2ec48ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.071515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b0755b86-12f9-4146-95bf-0f85b2ec48ae None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:28.073653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b0755b86-12f9-4146-95bf-0f85b2ec48ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:28.098733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:28.148831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:28.165909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4011ac-98a2-4787-8c40-9a3105245420 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.195833 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 313/623] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:28.197989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0755b86-12f9-4146-95bf-0f85b2ec48ae None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.217219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e228a-998e-49c3-a0ff-a073c2daaadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.218300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e228a-998e-49c3-a0ff-a073c2daaadc None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.218815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e228a-998e-49c3-a0ff-a073c2daaadc None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.220157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e228a-998e-49c3-a0ff-a073c2daaadc None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.221292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e228a-998e-49c3-a0ff-a073c2daaadc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:28.222616 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e228a-998e-49c3-a0ff-a073c2daaadc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.227294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e228a-998e-49c3-a0ff-a073c2daaadc None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.251862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 311/624] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:28.260891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e228a-998e-49c3-a0ff-a073c2daaadc None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.426484 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0755b86-12f9-4146-95bf-0f85b2ec48ae None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.475239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6da1d-2ee0-4280-a96a-2ea6a40a6d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.475455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6da1d-2ee0-4280-a96a-2ea6a40a6d1c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.475675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6da1d-2ee0-4280-a96a-2ea6a40a6d1c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.475878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6da1d-2ee0-4280-a96a-2ea6a40a6d1c None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.476543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6da1d-2ee0-4280-a96a-2ea6a40a6d1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:28.476847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6da1d-2ee0-4280-a96a-2ea6a40a6d1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.478232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6da1d-2ee0-4280-a96a-2ea6a40a6d1c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.494657 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e228a-998e-49c3-a0ff-a073c2daaadc None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.509108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 314/625] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:28.511211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:28.517119 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6da1d-2ee0-4280-a96a-2ea6a40a6d1c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.555863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed542672-9fd8-4403-8062-a3d28b7ce591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.556511 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed542672-9fd8-4403-8062-a3d28b7ce591 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.557328 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed542672-9fd8-4403-8062-a3d28b7ce591 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.557605 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed542672-9fd8-4403-8062-a3d28b7ce591 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.566064 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed542672-9fd8-4403-8062-a3d28b7ce591 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.567582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed542672-9fd8-4403-8062-a3d28b7ce591 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-ed542672-9fd8-4403-8062-a3d28b7ce591 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:28.580893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:28.581915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:28.582859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:28.582859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:28.582859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:28.582859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:28.582859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:28.582859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:28.582859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:28.582859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:28.582859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:35:28.582859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:28.582859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 312/626] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:28.586453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed542672-9fd8-4403-8062-a3d28b7ce591 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.595809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:28.691619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6da1d-2ee0-4280-a96a-2ea6a40a6d1c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.733966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d73cf-20d0-4d27-96d3-02dcd9ed6ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.734982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d73cf-20d0-4d27-96d3-02dcd9ed6ce9 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.735536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d73cf-20d0-4d27-96d3-02dcd9ed6ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.736027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d73cf-20d0-4d27-96d3-02dcd9ed6ce9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.736133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed542672-9fd8-4403-8062-a3d28b7ce591 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.744346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d73cf-20d0-4d27-96d3-02dcd9ed6ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.746166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d73cf-20d0-4d27-96d3-02dcd9ed6ce9 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-148d73cf-20d0-4d27-96d3-02dcd9ed6ce9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:28.759214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:28.760830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:28.762557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:28.762557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:28.762557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:28.762557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:28.762557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:28.762557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:28.762557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:28.762557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:28.762557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 315/627] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:28.762557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f48762-b0e2-419b-8f7f-659f6d4f21d1 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.763792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f48762-b0e2-419b-8f7f-659f6d4f21d1 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.763792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f48762-b0e2-419b-8f7f-659f6d4f21d1 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.768655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 316/628] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:28.769573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f48762-b0e2-419b-8f7f-659f6d4f21d1 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.800314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c70ea-c4a6-457c-a729-0c2bd40c1a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.800959 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c70ea-c4a6-457c-a729-0c2bd40c1a4e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.801265 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c70ea-c4a6-457c-a729-0c2bd40c1a4e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.801990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c70ea-c4a6-457c-a729-0c2bd40c1a4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.816717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c70ea-c4a6-457c-a729-0c2bd40c1a4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-367c70ea-c4a6-457c-a729-0c2bd40c1a4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:28.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:28.823768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:28.824589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 313/629] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:28.825271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c70ea-c4a6-457c-a729-0c2bd40c1a4e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.879418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:28.913218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:28.919369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f48762-b0e2-419b-8f7f-659f6d4f21d1 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.964140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f125e71-2bcd-4807-ae8e-79bbc1358f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.965026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f125e71-2bcd-4807-ae8e-79bbc1358f7a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.965315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f125e71-2bcd-4807-ae8e-79bbc1358f7a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.965744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f125e71-2bcd-4807-ae8e-79bbc1358f7a None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.966882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f125e71-2bcd-4807-ae8e-79bbc1358f7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:28.967283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f125e71-2bcd-4807-ae8e-79bbc1358f7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.968856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f125e71-2bcd-4807-ae8e-79bbc1358f7a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.971505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c70ea-c4a6-457c-a729-0c2bd40c1a4e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.998303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 317/630] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Aug 31 09:35:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:28.999617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265df5f9-c154-4d83-9f0a-cd8d71b889fe None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.999970 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265df5f9-c154-4d83-9f0a-cd8d71b889fe None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.000321 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265df5f9-c154-4d83-9f0a-cd8d71b889fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.027235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-265df5f9-c154-4d83-9f0a-cd8d71b889fe None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:29.028621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-265df5f9-c154-4d83-9f0a-cd8d71b889fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:29.030683 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c2fc8-b7ef-42dc-b1f8-c02177f99eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:29.031534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c2fc8-b7ef-42dc-b1f8-c02177f99eff None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:29.031962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c2fc8-b7ef-42dc-b1f8-c02177f99eff None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.032420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c2fc8-b7ef-42dc-b1f8-c02177f99eff None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.033245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c2fc8-b7ef-42dc-b1f8-c02177f99eff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:29.033554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c2fc8-b7ef-42dc-b1f8-c02177f99eff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:29.035097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c2fc8-b7ef-42dc-b1f8-c02177f99eff None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:29.051221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 314/631] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:29.052084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c2fc8-b7ef-42dc-b1f8-c02177f99eff None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:29.121766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 318/632] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:29.122908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265df5f9-c154-4d83-9f0a-cd8d71b889fe None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:29.179550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:29.198059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c2fc8-b7ef-42dc-b1f8-c02177f99eff None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:29.249811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa266a-7d05-4f31-a41d-7191d9b693c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:29.250427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa266a-7d05-4f31-a41d-7191d9b693c1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:29.250798 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa266a-7d05-4f31-a41d-7191d9b693c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.251073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa266a-7d05-4f31-a41d-7191d9b693c1 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.251818 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa266a-7d05-4f31-a41d-7191d9b693c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:29.252121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa266a-7d05-4f31-a41d-7191d9b693c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:29.253644 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa266a-7d05-4f31-a41d-7191d9b693c1 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:29.264825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 315/633] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:29.266375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a5f51-eb1e-4e42-9deb-7d74f05fd933 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:29.266683 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a5f51-eb1e-4e42-9deb-7d74f05fd933 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.266962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a5f51-eb1e-4e42-9deb-7d74f05fd933 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.268706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 316/634] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:29.272445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a5f51-eb1e-4e42-9deb-7d74f05fd933 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:29.276761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:29.298089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265df5f9-c154-4d83-9f0a-cd8d71b889fe None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:29.359231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b058418-8356-415e-ba23-318eb6304702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:29.360196 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b058418-8356-415e-ba23-318eb6304702 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:29.360695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b058418-8356-415e-ba23-318eb6304702 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.361151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b058418-8356-415e-ba23-318eb6304702 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.435877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a5f51-eb1e-4e42-9deb-7d74f05fd933 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:29.454779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b058418-8356-415e-ba23-318eb6304702 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:29.456770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b058418-8356-415e-ba23-318eb6304702 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:29.479213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c9370-f593-4c35-ab22-e0a67bf52319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:29.480054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c9370-f593-4c35-ab22-e0a67bf52319 None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:29.480337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c9370-f593-4c35-ab22-e0a67bf52319 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.480603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c9370-f593-4c35-ab22-e0a67bf52319 None admin] PATH_INFO: `/v3/projects/8ef8b586382941408cd88ee654c42d32/users/c5cf86ae159749cea0cc9783bffbfe75/roles/dcba9d07b31a4387b3014365e6e2ea10` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.497548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c9370-f593-4c35-ab22-e0a67bf52319 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef8b586382941408cd88ee654c42d32, user_id=c5cf86ae159749cea0cc9783bffbfe75, role_id=dcba9d07b31a4387b3014365e6e2ea10)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:29.498994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c9370-f593-4c35-ab22-e0a67bf52319 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:29.514763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 317/635] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:29 2022] PUT /identity/v3/projects/8ef8b586382941408cd88ee654c42d32/users/c5cf86ae159749cea0cc9783bffbfe75/roles/dcba9d07b31a4387b3014365e6e2ea10 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:35:29.515768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c9370-f593-4c35-ab22-e0a67bf52319 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:29.519860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:29.578759 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:29.604411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 319/636] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:29.604411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ffe30-87bf-4bc7-9c66-2f5928e47066 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:29.604411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ffe30-87bf-4bc7-9c66-2f5928e47066 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.604411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ffe30-87bf-4bc7-9c66-2f5928e47066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.630470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-f27ffe30-87bf-4bc7-9c66-2f5928e47066 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:29.632190 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-f27ffe30-87bf-4bc7-9c66-2f5928e47066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:29.664420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c9370-f593-4c35-ab22-e0a67bf52319 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:29.707819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5a796-99a8-491d-af09-6787f65f254f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:29.708999 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5a796-99a8-491d-af09-6787f65f254f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:29.709514 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5a796-99a8-491d-af09-6787f65f254f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.710040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5a796-99a8-491d-af09-6787f65f254f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.724954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5a796-99a8-491d-af09-6787f65f254f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:29.726430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5a796-99a8-491d-af09-6787f65f254f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-75b5a796-99a8-491d-af09-6787f65f254f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:29.743809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:29.744862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:29.745913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:29.745913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:29.745913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:29.745913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 318/637] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:29.746148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5a796-99a8-491d-af09-6787f65f254f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:29.752697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 320/638] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:29.753888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ffe30-87bf-4bc7-9c66-2f5928e47066 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:29.820250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:29.887146 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5a796-99a8-491d-af09-6787f65f254f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:29.889055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ffe30-87bf-4bc7-9c66-2f5928e47066 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:29.921306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:29.935142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2b5e1-4fa4-4114-ad7d-192d58a817ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:29.936076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2b5e1-4fa4-4114-ad7d-192d58a817ba None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:29.936900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2b5e1-4fa4-4114-ad7d-192d58a817ba None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.937327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2b5e1-4fa4-4114-ad7d-192d58a817ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.951762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2b5e1-4fa4-4114-ad7d-192d58a817ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:29.954891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40733b6b-9bf2-4a8e-b6c1-cd790d566322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:29.956298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40733b6b-9bf2-4a8e-b6c1-cd790d566322 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:29.956920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40733b6b-9bf2-4a8e-b6c1-cd790d566322 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.957577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40733b6b-9bf2-4a8e-b6c1-cd790d566322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-d3f2b5e1-4fa4-4114-ad7d-192d58a817ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:29.958729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:29.959814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:29.960835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:29.960835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:29.960835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:29.960835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:29.960835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:29.960835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:29.960835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:29.960835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:29.960835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:29.960835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:29.960835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:29.960835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:29.968430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 319/639] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:29.969410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2b5e1-4fa4-4114-ad7d-192d58a817ba None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:29.982454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40733b6b-9bf2-4a8e-b6c1-cd790d566322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:29.984037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40733b6b-9bf2-4a8e-b6c1-cd790d566322 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-40733b6b-9bf2-4a8e-b6c1-cd790d566322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:30.002712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:30.003746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:30.004628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:30.004628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:30.004628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:30.009343 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 321/640] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:30.011192 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40733b6b-9bf2-4a8e-b6c1-cd790d566322 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.108472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2b5e1-4fa4-4114-ad7d-192d58a817ba None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.126598 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:30.143677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40733b6b-9bf2-4a8e-b6c1-cd790d566322 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.157432 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9fcf0-dfd3-4fa6-bdf7-f7d8e73b93fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.158143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9fcf0-dfd3-4fa6-bdf7-f7d8e73b93fc None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.158427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9fcf0-dfd3-4fa6-bdf7-f7d8e73b93fc None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.158692 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9fcf0-dfd3-4fa6-bdf7-f7d8e73b93fc None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.159390 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9fcf0-dfd3-4fa6-bdf7-f7d8e73b93fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:30.159688 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9fcf0-dfd3-4fa6-bdf7-f7d8e73b93fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:30.161111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9fcf0-dfd3-4fa6-bdf7-f7d8e73b93fc None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:30.175694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 320/641] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:30.176441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9fcf0-dfd3-4fa6-bdf7-f7d8e73b93fc None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.189879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad895a7-34f8-4b0f-9f94-440f10686135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.190502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad895a7-34f8-4b0f-9f94-440f10686135 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.190783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad895a7-34f8-4b0f-9f94-440f10686135 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.191056 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad895a7-34f8-4b0f-9f94-440f10686135 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.191787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad895a7-34f8-4b0f-9f94-440f10686135 None admin] RBAC: Adding query filter params (name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:30.192094 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad895a7-34f8-4b0f-9f94-440f10686135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:30.193506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad895a7-34f8-4b0f-9f94-440f10686135 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:30.204244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 322/642] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:30.205078 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad895a7-34f8-4b0f-9f94-440f10686135 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.246250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:30.297328 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9fcf0-dfd3-4fa6-bdf7-f7d8e73b93fc None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.334202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad895a7-34f8-4b0f-9f94-440f10686135 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.345358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b361139-2942-417f-bb83-db9ae374ca01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.345987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b361139-2942-417f-bb83-db9ae374ca01 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.346270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b361139-2942-417f-bb83-db9ae374ca01 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.346547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b361139-2942-417f-bb83-db9ae374ca01 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.347236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b361139-2942-417f-bb83-db9ae374ca01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:30.347542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b361139-2942-417f-bb83-db9ae374ca01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:30.348938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b361139-2942-417f-bb83-db9ae374ca01 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:30.371373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 321/643] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:30.381106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b361139-2942-417f-bb83-db9ae374ca01 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.389056 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bb7d9-9d82-4950-bd4e-227904d39778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.390761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bb7d9-9d82-4950-bd4e-227904d39778 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.391429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bb7d9-9d82-4950-bd4e-227904d39778 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.391936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bb7d9-9d82-4950-bd4e-227904d39778 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.409473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bb7d9-9d82-4950-bd4e-227904d39778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-236bb7d9-9d82-4950-bd4e-227904d39778 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:30.419893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:30.421780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:30.423509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:30.423509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:30.423509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:30.423509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:30.423509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:30.423509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:30.423509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:30.423509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:30.423509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:30.423509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:30.423509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:30.423509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:30.424388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 323/644] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:30.425464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bb7d9-9d82-4950-bd4e-227904d39778 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.426218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:30.534061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b361139-2942-417f-bb83-db9ae374ca01 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.550260 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:30.563413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bb7d9-9d82-4950-bd4e-227904d39778 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.582764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18832504-79d0-49ad-ab27-355f677d581a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.583705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18832504-79d0-49ad-ab27-355f677d581a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.583972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18832504-79d0-49ad-ab27-355f677d581a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.584285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18832504-79d0-49ad-ab27-355f677d581a None admin] PATH_INFO: `/v3/users/nova` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.593301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18832504-79d0-49ad-ab27-355f677d581a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:30.594739 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18832504-79d0-49ad-ab27-355f677d581a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-18832504-79d0-49ad-ab27-355f677d581a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:30.606809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:30.607846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:30.613650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:30.613650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:30.613650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:30.613650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:30.613650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:30.613650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:30.613650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:30.613650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:30.613650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:30.613650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:30.613650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 322/645] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:30.613650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18832504-79d0-49ad-ab27-355f677d581a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.629489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24d1d1-2965-469e-89d1-72cdad2b0df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.630612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24d1d1-2965-469e-89d1-72cdad2b0df1 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.631182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24d1d1-2965-469e-89d1-72cdad2b0df1 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.631708 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24d1d1-2965-469e-89d1-72cdad2b0df1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.651074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24d1d1-2965-469e-89d1-72cdad2b0df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:30.653810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24d1d1-2965-469e-89d1-72cdad2b0df1 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-8b24d1d1-2965-469e-89d1-72cdad2b0df1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:30.671517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:30.672508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:30.673396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:30.673396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:30.673396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:30.680268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 324/646] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:30.680268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24d1d1-2965-469e-89d1-72cdad2b0df1 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.723397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:30.732523 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18832504-79d0-49ad-ab27-355f677d581a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.773120 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082d700-3203-479a-9255-e52f882b43e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.773810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082d700-3203-479a-9255-e52f882b43e2 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.774190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082d700-3203-479a-9255-e52f882b43e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.774546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082d700-3203-479a-9255-e52f882b43e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.783182 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082d700-3203-479a-9255-e52f882b43e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:30.784787 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082d700-3203-479a-9255-e52f882b43e2 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-b082d700-3203-479a-9255-e52f882b43e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:30.798318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:30.800197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:30.801987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:30.801987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:30.801987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:30.801987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:30.801987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:30.801987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:30.801987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:30.801987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:30.804106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 323/647] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:30.804924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082d700-3203-479a-9255-e52f882b43e2 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.824698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24d1d1-2965-469e-89d1-72cdad2b0df1 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.879654 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7dbffc-eeef-43dd-824c-9e4d7cd6357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.880575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dbffc-eeef-43dd-824c-9e4d7cd6357a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.881078 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dbffc-eeef-43dd-824c-9e4d7cd6357a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.881514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dbffc-eeef-43dd-824c-9e4d7cd6357a None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.882689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7dbffc-eeef-43dd-824c-9e4d7cd6357a None admin] RBAC: Adding query filter params (name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:30.883101 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7dbffc-eeef-43dd-824c-9e4d7cd6357a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:30.886271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7dbffc-eeef-43dd-824c-9e4d7cd6357a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:30.904980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 325/648] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:30.904980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7dbffc-eeef-43dd-824c-9e4d7cd6357a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.917627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:30.936796 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082d700-3203-479a-9255-e52f882b43e2 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.979103 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb39d4-1ded-4fd4-bb37-ce5638925a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.980534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb39d4-1ded-4fd4-bb37-ce5638925a18 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.980834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb39d4-1ded-4fd4-bb37-ce5638925a18 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.981076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb39d4-1ded-4fd4-bb37-ce5638925a18 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.981805 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb39d4-1ded-4fd4-bb37-ce5638925a18 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:30.982106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb39d4-1ded-4fd4-bb37-ce5638925a18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:30.983571 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb39d4-1ded-4fd4-bb37-ce5638925a18 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.011733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 324/649] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:31.013538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb39d4-1ded-4fd4-bb37-ce5638925a18 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.040756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:31.071105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7dbffc-eeef-43dd-824c-9e4d7cd6357a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.130799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b8485-922c-43c1-b1d7-bb1b82857744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.131792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b8485-922c-43c1-b1d7-bb1b82857744 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.132295 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b8485-922c-43c1-b1d7-bb1b82857744 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.132774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b8485-922c-43c1-b1d7-bb1b82857744 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.133520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b8485-922c-43c1-b1d7-bb1b82857744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:31.134075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b8485-922c-43c1-b1d7-bb1b82857744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.137252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b8485-922c-43c1-b1d7-bb1b82857744 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.154995 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb39d4-1ded-4fd4-bb37-ce5638925a18 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.163593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 326/650] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:31.163593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b8485-922c-43c1-b1d7-bb1b82857744 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.196321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a887a63-4fbc-4b2a-a0fe-da63b355b12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.196934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a887a63-4fbc-4b2a-a0fe-da63b355b12e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.197874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a887a63-4fbc-4b2a-a0fe-da63b355b12e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.198285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a887a63-4fbc-4b2a-a0fe-da63b355b12e None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.199341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a887a63-4fbc-4b2a-a0fe-da63b355b12e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:31.199648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a887a63-4fbc-4b2a-a0fe-da63b355b12e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.201214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a887a63-4fbc-4b2a-a0fe-da63b355b12e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.219946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 325/651] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:31.219946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a887a63-4fbc-4b2a-a0fe-da63b355b12e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.277737 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:31.315210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:31.337589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b8485-922c-43c1-b1d7-bb1b82857744 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.359303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a887a63-4fbc-4b2a-a0fe-da63b355b12e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.384034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80ce76-e3b0-4b22-ad3f-ceac1a0bd437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.386448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80ce76-e3b0-4b22-ad3f-ceac1a0bd437 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.387203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80ce76-e3b0-4b22-ad3f-ceac1a0bd437 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.387653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80ce76-e3b0-4b22-ad3f-ceac1a0bd437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.401516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80ce76-e3b0-4b22-ad3f-ceac1a0bd437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.403632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80ce76-e3b0-4b22-ad3f-ceac1a0bd437 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.408460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e87cd-6ccd-4160-b189-bf9fb1850469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.409510 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e87cd-6ccd-4160-b189-bf9fb1850469 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.410004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e87cd-6ccd-4160-b189-bf9fb1850469 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.410483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e87cd-6ccd-4160-b189-bf9fb1850469 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-5a80ce76-e3b0-4b22-ad3f-ceac1a0bd437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:31.421919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:31.423115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:31.424219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:31.424219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:31.424219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:31.428004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 327/652] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:31.428813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80ce76-e3b0-4b22-ad3f-ceac1a0bd437 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.517136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e87cd-6ccd-4160-b189-bf9fb1850469 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.520046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e87cd-6ccd-4160-b189-bf9fb1850469 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.578696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80ce76-e3b0-4b22-ad3f-ceac1a0bd437 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.597296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:31.609388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:31.618989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d65b4-6cfb-4a4f-b9a9-7121800c1dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.620275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d65b4-6cfb-4a4f-b9a9-7121800c1dc7 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.620551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d65b4-6cfb-4a4f-b9a9-7121800c1dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.620827 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d65b4-6cfb-4a4f-b9a9-7121800c1dc7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.629196 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d65b4-6cfb-4a4f-b9a9-7121800c1dc7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.630648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d65b4-6cfb-4a4f-b9a9-7121800c1dc7 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.642825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 326/653] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-377d65b4-6cfb-4a4f-b9a9-7121800c1dc7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:31.643238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:31.644158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:31.645699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:31.645699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:31.645699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:31.645699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:31.645699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:31.645699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:31.645699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:31.645699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:31.645699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:35:31.645699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:31.646278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e87cd-6ccd-4160-b189-bf9fb1850469 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.646349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 328/654] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:31.646349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d65b4-6cfb-4a4f-b9a9-7121800c1dc7 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.754504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d65b4-6cfb-4a4f-b9a9-7121800c1dc7 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.754934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e87cd-6ccd-4160-b189-bf9fb1850469 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.791817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d2c3a-a557-48bc-ba05-9a5c4d78f3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.792159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563538b8-3845-40f7-a4c0-691924866a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.792424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d2c3a-a557-48bc-ba05-9a5c4d78f3da None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.792579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563538b8-3845-40f7-a4c0-691924866a4d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.792722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563538b8-3845-40f7-a4c0-691924866a4d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.792843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d2c3a-a557-48bc-ba05-9a5c4d78f3da None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.792971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563538b8-3845-40f7-a4c0-691924866a4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.793117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d2c3a-a557-48bc-ba05-9a5c4d78f3da None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.793688 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d2c3a-a557-48bc-ba05-9a5c4d78f3da None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:31.793824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563538b8-3845-40f7-a4c0-691924866a4d None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=06fad9466df14827983dac2b4b844f3a) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:31.793979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563538b8-3845-40f7-a4c0-691924866a4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.794103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d2c3a-a557-48bc-ba05-9a5c4d78f3da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.795366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d2c3a-a557-48bc-ba05-9a5c4d78f3da None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.795498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563538b8-3845-40f7-a4c0-691924866a4d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.801834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 327/655] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/role_assignments?user.id=06fad9466df14827983dac2b4b844f3a&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:31.802997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a03928-3447-483a-9033-2e5ab1cd56f6 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.803366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a03928-3447-483a-9033-2e5ab1cd56f6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.803632 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a03928-3447-483a-9033-2e5ab1cd56f6 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.804926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 328/656] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:31.805660 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a03928-3447-483a-9033-2e5ab1cd56f6 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.824242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 329/657] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:31.826822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d2c3a-a557-48bc-ba05-9a5c4d78f3da None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.853673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:31.904255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:31.982383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a03928-3447-483a-9033-2e5ab1cd56f6 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.988990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d2c3a-a557-48bc-ba05-9a5c4d78f3da None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:32.037573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b2e38-1f74-4426-b102-0a935de14309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:32.038777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b2e38-1f74-4426-b102-0a935de14309 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.039082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b2e38-1f74-4426-b102-0a935de14309 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.039703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b2e38-1f74-4426-b102-0a935de14309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.045424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314838b7-cb68-48d5-b950-d8616c84f38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:32.046058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314838b7-cb68-48d5-b950-d8616c84f38b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.046334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314838b7-cb68-48d5-b950-d8616c84f38b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.046606 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314838b7-cb68-48d5-b950-d8616c84f38b None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.047422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314838b7-cb68-48d5-b950-d8616c84f38b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:32.049706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314838b7-cb68-48d5-b950-d8616c84f38b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:32.050287 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314838b7-cb68-48d5-b950-d8616c84f38b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:32.052829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b2e38-1f74-4426-b102-0a935de14309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:32.055542 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b2e38-1f74-4426-b102-0a935de14309 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:32.079792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 329/658] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:32.081075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eeb399-2dc8-4f98-a8ae-cc349e9b2fc2 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.081348 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eeb399-2dc8-4f98-a8ae-cc349e9b2fc2 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.081635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eeb399-2dc8-4f98-a8ae-cc349e9b2fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-332b2e38-1f74-4426-b102-0a935de14309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:32.088886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:32.089845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:32.090709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:32.090709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:32.090709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:32.095371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 330/659] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:32.096939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b2e38-1f74-4426-b102-0a935de14309 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:32.107475 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-79eeb399-2dc8-4f98-a8ae-cc349e9b2fc2 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:32.108702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-79eeb399-2dc8-4f98-a8ae-cc349e9b2fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:32.213278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 330/660] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:32.213278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eeb399-2dc8-4f98-a8ae-cc349e9b2fc2 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:32.249875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:32.268096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:32.285891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b2e38-1f74-4426-b102-0a935de14309 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:32.352534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6021af-6634-4343-99a4-1f5caaf41bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:32.353807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6021af-6634-4343-99a4-1f5caaf41bd6 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.354404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6021af-6634-4343-99a4-1f5caaf41bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.355013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6021af-6634-4343-99a4-1f5caaf41bd6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.370591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6021af-6634-4343-99a4-1f5caaf41bd6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:32.373515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6021af-6634-4343-99a4-1f5caaf41bd6 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:32.381352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eeb399-2dc8-4f98-a8ae-cc349e9b2fc2 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-9e6021af-6634-4343-99a4-1f5caaf41bd6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:32.397082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:32.399175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:32.401402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:32.401402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:32.401402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:32.401402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:32.401402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:32.401402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:32.401402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:32.401402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:32.401402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:35:32.401402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:32.409561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 331/661] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:32.409561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6021af-6634-4343-99a4-1f5caaf41bd6 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:32.436939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81a50e-62f0-401a-a733-d94f4206c912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:32.437550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a50e-62f0-401a-a733-d94f4206c912 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.437831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a50e-62f0-401a-a733-d94f4206c912 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.438091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a50e-62f0-401a-a733-d94f4206c912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.513246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81a50e-62f0-401a-a733-d94f4206c912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:32.514628 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81a50e-62f0-401a-a733-d94f4206c912 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:32.542322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:32.612820 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6021af-6634-4343-99a4-1f5caaf41bd6 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:32.624340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 331/662] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:32.625217 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81a50e-62f0-401a-a733-d94f4206c912 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:32.659719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33f56c-122a-4631-a3de-ab25abde51cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:32.661249 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33f56c-122a-4631-a3de-ab25abde51cc None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.662001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33f56c-122a-4631-a3de-ab25abde51cc None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.662488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33f56c-122a-4631-a3de-ab25abde51cc None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.663578 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33f56c-122a-4631-a3de-ab25abde51cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:32.664234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33f56c-122a-4631-a3de-ab25abde51cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:32.665939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33f56c-122a-4631-a3de-ab25abde51cc None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:32.685306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 332/663] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:32.686694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33f56c-122a-4631-a3de-ab25abde51cc None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:32.702972 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:32.750216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81a50e-62f0-401a-a733-d94f4206c912 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:32.788342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:32.808020 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f3346-221e-458b-943e-eddae1e56e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:32.809077 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f3346-221e-458b-943e-eddae1e56e7c None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.809573 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f3346-221e-458b-943e-eddae1e56e7c None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.810060 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f3346-221e-458b-943e-eddae1e56e7c None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.811029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f3346-221e-458b-943e-eddae1e56e7c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:32.811602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f3346-221e-458b-943e-eddae1e56e7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:32.814184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f3346-221e-458b-943e-eddae1e56e7c None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:32.815065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33f56c-122a-4631-a3de-ab25abde51cc None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:32.854782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 332/664] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Aug 31 09:35:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:32.856253 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f3346-221e-458b-943e-eddae1e56e7c None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:32.862684 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48e908-b244-4382-8c32-acb9caa81928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:32.863395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48e908-b244-4382-8c32-acb9caa81928 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.863703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48e908-b244-4382-8c32-acb9caa81928 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.864162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48e908-b244-4382-8c32-acb9caa81928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.876644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48e908-b244-4382-8c32-acb9caa81928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:32.878982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48e908-b244-4382-8c32-acb9caa81928 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-ba48e908-b244-4382-8c32-acb9caa81928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:32.905861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:32.907334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:32.908899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:32.908899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:32.908899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:32.908899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 333/665] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:32.909685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48e908-b244-4382-8c32-acb9caa81928 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:33.001567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f3346-221e-458b-943e-eddae1e56e7c None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:33.018538 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:33.053960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c8a6c-d40b-40e1-988c-67a5f55ce197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:33.054866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c8a6c-d40b-40e1-988c-67a5f55ce197 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.059866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c8a6c-d40b-40e1-988c-67a5f55ce197 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.059866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c8a6c-d40b-40e1-988c-67a5f55ce197 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.069403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c8a6c-d40b-40e1-988c-67a5f55ce197 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:33.071870 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c8a6c-d40b-40e1-988c-67a5f55ce197 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-4c7c8a6c-d40b-40e1-988c-67a5f55ce197 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:33.092362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:33.094100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:33.096104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:33.096104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:33.096104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:33.096104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:33.096104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:33.096104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:33.096104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:33.096104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:33.099356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 333/666] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:33.101167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c8a6c-d40b-40e1-988c-67a5f55ce197 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:33.103373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48e908-b244-4382-8c32-acb9caa81928 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:33.145372 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ff88d-e78e-4932-a0e4-40ecc7d80517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:33.146021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ff88d-e78e-4932-a0e4-40ecc7d80517 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.146302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ff88d-e78e-4932-a0e4-40ecc7d80517 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.146579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ff88d-e78e-4932-a0e4-40ecc7d80517 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.164637 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ff88d-e78e-4932-a0e4-40ecc7d80517 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:33.169151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-570ff88d-e78e-4932-a0e4-40ecc7d80517 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:33.171057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:33.172477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:33.173841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:33.173841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:33.173841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:33.173841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:33.173841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:33.173841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:33.173841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:33.173841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:33.173841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:33.173841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:33.173841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:33.173841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:33.177183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 334/667] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:33.178050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ff88d-e78e-4932-a0e4-40ecc7d80517 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:33.246973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c8a6c-d40b-40e1-988c-67a5f55ce197 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:33.292446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f9b3a-ef42-430c-8a14-a7725364c785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:33.293204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f9b3a-ef42-430c-8a14-a7725364c785 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.293557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f9b3a-ef42-430c-8a14-a7725364c785 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.293832 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f9b3a-ef42-430c-8a14-a7725364c785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.341238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ff88d-e78e-4932-a0e4-40ecc7d80517 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:33.377022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:33.396598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf3e68-68d1-400f-871a-23c34b2f4cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:33.397506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf3e68-68d1-400f-871a-23c34b2f4cf7 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.397781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf3e68-68d1-400f-871a-23c34b2f4cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.398063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf3e68-68d1-400f-871a-23c34b2f4cf7 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.398726 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf3e68-68d1-400f-871a-23c34b2f4cf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:33.399031 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf3e68-68d1-400f-871a-23c34b2f4cf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:33.401315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf3e68-68d1-400f-871a-23c34b2f4cf7 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:33.406789 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f9b3a-ef42-430c-8a14-a7725364c785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:33.410025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f9b3a-ef42-430c-8a14-a7725364c785 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:33.416318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 335/668] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:33.416497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf3e68-68d1-400f-871a-23c34b2f4cf7 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:33.485959 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:33.553546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf3e68-68d1-400f-871a-23c34b2f4cf7 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:33.576099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 334/669] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:33.577309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f9b3a-ef42-430c-8a14-a7725364c785 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:33.609429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3d476-0d4e-426e-b264-18a6578fc43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:33.610438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3d476-0d4e-426e-b264-18a6578fc43a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.610998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3d476-0d4e-426e-b264-18a6578fc43a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.611373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3d476-0d4e-426e-b264-18a6578fc43a None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.612086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3d476-0d4e-426e-b264-18a6578fc43a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:33.612383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3d476-0d4e-426e-b264-18a6578fc43a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:33.613871 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3d476-0d4e-426e-b264-18a6578fc43a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:33.638012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 336/670] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:33.638012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3d476-0d4e-426e-b264-18a6578fc43a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:33.697259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:33.722085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f9b3a-ef42-430c-8a14-a7725364c785 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:33.766957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2d9bb-eafc-4038-9ef8-8e1012e16e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:33.767624 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2d9bb-eafc-4038-9ef8-8e1012e16e7f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.767901 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2d9bb-eafc-4038-9ef8-8e1012e16e7f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.768176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2d9bb-eafc-4038-9ef8-8e1012e16e7f None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.768847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2d9bb-eafc-4038-9ef8-8e1012e16e7f None admin] RBAC: Adding query filter params (name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:33.769150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2d9bb-eafc-4038-9ef8-8e1012e16e7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:33.769736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3d476-0d4e-426e-b264-18a6578fc43a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:33.770535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2d9bb-eafc-4038-9ef8-8e1012e16e7f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:33.784292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 335/671] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:33.784292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2d9bb-eafc-4038-9ef8-8e1012e16e7f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:33.786477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:33.825020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2e57d-090f-4f31-979f-84d3ebfcb855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:33.825679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2e57d-090f-4f31-979f-84d3ebfcb855 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.825964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2e57d-090f-4f31-979f-84d3ebfcb855 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.826336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2e57d-090f-4f31-979f-84d3ebfcb855 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.835043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2e57d-090f-4f31-979f-84d3ebfcb855 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:33.836616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2e57d-090f-4f31-979f-84d3ebfcb855 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-d2d2e57d-090f-4f31-979f-84d3ebfcb855 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:33.849855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:33.851557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:33.854010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:33.854010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:33.854010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:33.854010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:33.854010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:33.854010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:33.854010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:33.854010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:33.854010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 337/672] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:33.854010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2e57d-090f-4f31-979f-84d3ebfcb855 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:33.925483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2d9bb-eafc-4038-9ef8-8e1012e16e7f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:33.971929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dd6eb-d056-4a86-9be3-c6bd2ecedba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:33.973179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dd6eb-d056-4a86-9be3-c6bd2ecedba3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.973650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dd6eb-d056-4a86-9be3-c6bd2ecedba3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.974125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dd6eb-d056-4a86-9be3-c6bd2ecedba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.992628 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dd6eb-d056-4a86-9be3-c6bd2ecedba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:33.996055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dd6eb-d056-4a86-9be3-c6bd2ecedba3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:33.999052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2e57d-090f-4f31-979f-84d3ebfcb855 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:34.000972 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-a44dd6eb-d056-4a86-9be3-c6bd2ecedba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:34.018493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:34.020126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:34.021057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:34.021057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:34.021057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:34.021057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 336/673] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:34.021465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dd6eb-d056-4a86-9be3-c6bd2ecedba3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:34.050584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3efad-0284-4820-9560-4d2093377f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:34.051677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3efad-0284-4820-9560-4d2093377f3d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.052135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3efad-0284-4820-9560-4d2093377f3d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.052628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3efad-0284-4820-9560-4d2093377f3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.054044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3efad-0284-4820-9560-4d2093377f3d None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=ef72bb95992c4a21ae2dbb2a08c51421) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:34.054638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3efad-0284-4820-9560-4d2093377f3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:34.057537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3efad-0284-4820-9560-4d2093377f3d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:34.068958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 338/674] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:33 2022] GET /identity/v3/role_assignments?user.id=ef72bb95992c4a21ae2dbb2a08c51421&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:34.070198 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3efad-0284-4820-9560-4d2093377f3d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:34.131595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:34.202052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dd6eb-d056-4a86-9be3-c6bd2ecedba3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:34.232742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3efad-0284-4820-9560-4d2093377f3d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:34.246897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4c26f-7e0e-4351-a551-57cf106f9a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:34.247622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4c26f-7e0e-4351-a551-57cf106f9a01 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.247896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4c26f-7e0e-4351-a551-57cf106f9a01 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.248160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4c26f-7e0e-4351-a551-57cf106f9a01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.262853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4c26f-7e0e-4351-a551-57cf106f9a01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:34.265838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4c26f-7e0e-4351-a551-57cf106f9a01 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:34.285238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c191ef6-8905-433e-98f9-cca9c3a9e611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:34.285863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c191ef6-8905-433e-98f9-cca9c3a9e611 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.286137 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c191ef6-8905-433e-98f9-cca9c3a9e611 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.286411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c191ef6-8905-433e-98f9-cca9c3a9e611 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.287113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c191ef6-8905-433e-98f9-cca9c3a9e611 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:34.287449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c191ef6-8905-433e-98f9-cca9c3a9e611 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:34.288899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c191ef6-8905-433e-98f9-cca9c3a9e611 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-4db4c26f-7e0e-4351-a551-57cf106f9a01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:34.296535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:34.297541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:34.298441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:34.298441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:34.298441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:34.299694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 337/675] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:34.300266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4c26f-7e0e-4351-a551-57cf106f9a01 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:34.307733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 339/676] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:34.309647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714b00c-a81f-4632-ac95-8c7129924720 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.309647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714b00c-a81f-4632-ac95-8c7129924720 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.309647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714b00c-a81f-4632-ac95-8c7129924720 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.311113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 340/677] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:34.312052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714b00c-a81f-4632-ac95-8c7129924720 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:34.360338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:34.454795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714b00c-a81f-4632-ac95-8c7129924720 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:34.454974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4c26f-7e0e-4351-a551-57cf106f9a01 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:34.471680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:34.510827 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7500725-6217-4850-bc0d-7449da21404c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:34.511822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7500725-6217-4850-bc0d-7449da21404c None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.512275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7500725-6217-4850-bc0d-7449da21404c None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.512727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7500725-6217-4850-bc0d-7449da21404c None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.513822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7500725-6217-4850-bc0d-7449da21404c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:34.514299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7500725-6217-4850-bc0d-7449da21404c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:34.514440 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2161f-6c02-42f8-9c66-93fdd754fd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:34.515619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2161f-6c02-42f8-9c66-93fdd754fd67 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.516035 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2161f-6c02-42f8-9c66-93fdd754fd67 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.516463 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2161f-6c02-42f8-9c66-93fdd754fd67 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.516934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7500725-6217-4850-bc0d-7449da21404c None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:34.517509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2161f-6c02-42f8-9c66-93fdd754fd67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:34.517978 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2161f-6c02-42f8-9c66-93fdd754fd67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:34.520165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2161f-6c02-42f8-9c66-93fdd754fd67 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:34.542569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 338/678] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:34.542569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb44af-85c9-4300-a78f-5f43b1b76d64 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.542569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb44af-85c9-4300-a78f-5f43b1b76d64 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.542569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb44af-85c9-4300-a78f-5f43b1b76d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.547540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 341/679] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:34.548963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2161f-6c02-42f8-9c66-93fdd754fd67 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:34.567877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4fdb44af-85c9-4300-a78f-5f43b1b76d64 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:34.569679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4fdb44af-85c9-4300-a78f-5f43b1b76d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:34.669964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 339/680] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:34.671640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb44af-85c9-4300-a78f-5f43b1b76d64 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:34.698667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:34.721075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2161f-6c02-42f8-9c66-93fdd754fd67 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:34.762585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61617c06-171b-4090-99ad-5a7b4bbe1b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:34.763559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61617c06-171b-4090-99ad-5a7b4bbe1b6b None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.764081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61617c06-171b-4090-99ad-5a7b4bbe1b6b None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.764559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61617c06-171b-4090-99ad-5a7b4bbe1b6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.765751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61617c06-171b-4090-99ad-5a7b4bbe1b6b None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=6ff1177119cb48e0a88b0f11a6b1b0c3) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:34.766428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61617c06-171b-4090-99ad-5a7b4bbe1b6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:34.768095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61617c06-171b-4090-99ad-5a7b4bbe1b6b None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:34.775865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 342/681] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:34 2022] GET /identity/v3/role_assignments?user.id=6ff1177119cb48e0a88b0f11a6b1b0c3&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:34.777238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61617c06-171b-4090-99ad-5a7b4bbe1b6b None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:34.844930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:34.850862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb44af-85c9-4300-a78f-5f43b1b76d64 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:34.926331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dbb49-27d0-4d96-9045-4e9ae885c843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:34.927149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dbb49-27d0-4d96-9045-4e9ae885c843 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.927445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dbb49-27d0-4d96-9045-4e9ae885c843 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.927720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dbb49-27d0-4d96-9045-4e9ae885c843 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.936835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dbb49-27d0-4d96-9045-4e9ae885c843 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:34.938880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dbb49-27d0-4d96-9045-4e9ae885c843 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-b21dbb49-27d0-4d96-9045-4e9ae885c843 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:34.955896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:34.956853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:34.957728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:34.957728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:34.957728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:34.957728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:34.957728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:34.957728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:34.957728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:34.957728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:34.958152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 340/682] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:34.965506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dbb49-27d0-4d96-9045-4e9ae885c843 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:35.028215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61617c06-171b-4090-99ad-5a7b4bbe1b6b None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:35.073842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87bd7e-e353-4f5b-b350-0df2b86d12bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:35.074946 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87bd7e-e353-4f5b-b350-0df2b86d12bc None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:35.075470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87bd7e-e353-4f5b-b350-0df2b86d12bc None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:35.075959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87bd7e-e353-4f5b-b350-0df2b86d12bc None admin] PATH_INFO: `/v3/users/swift` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:35.094380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87bd7e-e353-4f5b-b350-0df2b86d12bc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:35.096527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87bd7e-e353-4f5b-b350-0df2b86d12bc None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-7d87bd7e-e353-4f5b-b350-0df2b86d12bc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:35.113515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:35.114574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:35.117261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:35.117261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:35.117261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:35.117261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:35.117261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:35.117261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:35.117261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:35.117261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:35.117261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:35.117261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:35.120858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 343/683] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:35.131278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87bd7e-e353-4f5b-b350-0df2b86d12bc None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:35.150805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:35.185165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dbb49-27d0-4d96-9045-4e9ae885c843 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:35.258013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2abbab-f707-4405-93e4-01a8bd1bc867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:35.259431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2abbab-f707-4405-93e4-01a8bd1bc867 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:35.260605 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2abbab-f707-4405-93e4-01a8bd1bc867 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:35.261078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2abbab-f707-4405-93e4-01a8bd1bc867 None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:35.262355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2abbab-f707-4405-93e4-01a8bd1bc867 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:35.262997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2abbab-f707-4405-93e4-01a8bd1bc867 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:35.267236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2abbab-f707-4405-93e4-01a8bd1bc867 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:35.286596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 341/684] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:35.287500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2abbab-f707-4405-93e4-01a8bd1bc867 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:35.291965 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87bd7e-e353-4f5b-b350-0df2b86d12bc None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:35.317687 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:35.373504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7b2fe-d43f-4d40-afb2-760be7d73c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:35.374922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7b2fe-d43f-4d40-afb2-760be7d73c91 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:35.375456 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7b2fe-d43f-4d40-afb2-760be7d73c91 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:35.376289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7b2fe-d43f-4d40-afb2-760be7d73c91 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:35.378620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7b2fe-d43f-4d40-afb2-760be7d73c91 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:35.379163 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7b2fe-d43f-4d40-afb2-760be7d73c91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:35.382260 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7b2fe-d43f-4d40-afb2-760be7d73c91 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:35.404723 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 344/685] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Aug 31 09:35:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:35.405835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7b2fe-d43f-4d40-afb2-760be7d73c91 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:35.456694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2abbab-f707-4405-93e4-01a8bd1bc867 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:35.477420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:35.530833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534a429-cb05-4a89-ae9e-bcdc72dd7864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:35.532083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534a429-cb05-4a89-ae9e-bcdc72dd7864 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:35.532557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534a429-cb05-4a89-ae9e-bcdc72dd7864 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:35.533123 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534a429-cb05-4a89-ae9e-bcdc72dd7864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:35.537415 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7b2fe-d43f-4d40-afb2-760be7d73c91 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:35.553467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534a429-cb05-4a89-ae9e-bcdc72dd7864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:35.559715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534a429-cb05-4a89-ae9e-bcdc72dd7864 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-7534a429-cb05-4a89-ae9e-bcdc72dd7864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:35.590366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:35.592245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:35.593968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:35.593968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:35.593968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:35.593968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 342/686] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:35.593968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534a429-cb05-4a89-ae9e-bcdc72dd7864 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:35.611634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f32a8-3232-471e-a813-8459984f184a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:35.612500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f32a8-3232-471e-a813-8459984f184a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:35.612779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f32a8-3232-471e-a813-8459984f184a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:35.613057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f32a8-3232-471e-a813-8459984f184a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:35.613977 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f32a8-3232-471e-a813-8459984f184a None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=21d8ade66b7e4a1880c961c5e13adf04) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:35.614319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f32a8-3232-471e-a813-8459984f184a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:35.616104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f32a8-3232-471e-a813-8459984f184a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:35.630448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 345/687] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:35 2022] GET /identity/v3/role_assignments?user.id=21d8ade66b7e4a1880c961c5e13adf04&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:35.631509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f32a8-3232-471e-a813-8459984f184a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:35.735047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:35.807612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534a429-cb05-4a89-ae9e-bcdc72dd7864 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:35.870778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3048e6-673a-4204-8c9e-dea8a0036cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:35.872407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3048e6-673a-4204-8c9e-dea8a0036cbd None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:35.873197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3048e6-673a-4204-8c9e-dea8a0036cbd None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:35.873891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3048e6-673a-4204-8c9e-dea8a0036cbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:35.888200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f32a8-3232-471e-a813-8459984f184a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:35.954528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:35.962429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a4e60-660e-4ab6-b9e9-5b9da97b8f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:35.963651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a4e60-660e-4ab6-b9e9-5b9da97b8f7e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:35.963929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a4e60-660e-4ab6-b9e9-5b9da97b8f7e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:35.964207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a4e60-660e-4ab6-b9e9-5b9da97b8f7e None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:35.965073 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a4e60-660e-4ab6-b9e9-5b9da97b8f7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:35.965365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a4e60-660e-4ab6-b9e9-5b9da97b8f7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:35.966835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a4e60-660e-4ab6-b9e9-5b9da97b8f7e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:35.983479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 346/688] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:35.992265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a4e60-660e-4ab6-b9e9-5b9da97b8f7e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:36.017147 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3048e6-673a-4204-8c9e-dea8a0036cbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:36.019960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3048e6-673a-4204-8c9e-dea8a0036cbd None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:36.163212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:36.178469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a4e60-660e-4ab6-b9e9-5b9da97b8f7e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:36.219903 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f53c36-cbf4-4129-88d5-9537f6bd3cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:36.220907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f53c36-cbf4-4129-88d5-9537f6bd3cd4 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:36.221406 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f53c36-cbf4-4129-88d5-9537f6bd3cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:36.221895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f53c36-cbf4-4129-88d5-9537f6bd3cd4 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:36.222823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f53c36-cbf4-4129-88d5-9537f6bd3cd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:36.224650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f53c36-cbf4-4129-88d5-9537f6bd3cd4 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:36.226322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 343/689] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:36.243165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3048e6-673a-4204-8c9e-dea8a0036cbd None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:36.263033 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 347/690] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:35:35 2022] POST /identity/v3/users => generated 254 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:36.319157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88613289-5bd2-4212-91df-e8b272a50a9e None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:36.320192 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88613289-5bd2-4212-91df-e8b272a50a9e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:36.320924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88613289-5bd2-4212-91df-e8b272a50a9e None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:36.329888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 348/691] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:36 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:36.337049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678526a-6bcd-4bf2-a736-68d2402602d1 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:36.338573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678526a-6bcd-4bf2-a736-68d2402602d1 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:36.339400 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678526a-6bcd-4bf2-a736-68d2402602d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:36.408347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-1678526a-6bcd-4bf2-a736-68d2402602d1 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:36.409599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1678526a-6bcd-4bf2-a736-68d2402602d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:36.452931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3048e6-673a-4204-8c9e-dea8a0036cbd None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:36.507423 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:36.523535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb68e7-026c-4d2e-9a2e-e1818e974b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:36.524523 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb68e7-026c-4d2e-9a2e-e1818e974b8c None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:36.524985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb68e7-026c-4d2e-9a2e-e1818e974b8c None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:36.525475 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb68e7-026c-4d2e-9a2e-e1818e974b8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:36.545652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb68e7-026c-4d2e-9a2e-e1818e974b8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:36.548250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb68e7-026c-4d2e-9a2e-e1818e974b8c None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:36.552982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 349/692] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-5dcb68e7-026c-4d2e-9a2e-e1818e974b8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:36.570745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:36.572829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:36.573738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:36.573738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:36.573738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:36.573738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 344/693] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:36.581880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678526a-6bcd-4bf2-a736-68d2402602d1 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:36.769216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678526a-6bcd-4bf2-a736-68d2402602d1 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:36.833940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d31f0-0933-465e-86d8-8c8f6cb11379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:36.834799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d31f0-0933-465e-86d8-8c8f6cb11379 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:36.835337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d31f0-0933-465e-86d8-8c8f6cb11379 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:36.835763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d31f0-0933-465e-86d8-8c8f6cb11379 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:36.836729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d31f0-0933-465e-86d8-8c8f6cb11379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:36.837279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d31f0-0933-465e-86d8-8c8f6cb11379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:36.841149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d31f0-0933-465e-86d8-8c8f6cb11379 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:36.864564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 350/694] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:36.872716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb68e7-026c-4d2e-9a2e-e1818e974b8c None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:36.873027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d31f0-0933-465e-86d8-8c8f6cb11379 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:36.935532 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:36.969053 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:37.142869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d31f0-0933-465e-86d8-8c8f6cb11379 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:37.144094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb68e7-026c-4d2e-9a2e-e1818e974b8c None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:37.187746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b4252-1922-4eb5-8c53-8ec5038d18a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:37.187746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b4252-1922-4eb5-8c53-8ec5038d18a0 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.187746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b4252-1922-4eb5-8c53-8ec5038d18a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.187746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b4252-1922-4eb5-8c53-8ec5038d18a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.196796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b4252-1922-4eb5-8c53-8ec5038d18a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:37.199009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b4252-1922-4eb5-8c53-8ec5038d18a0 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-e86b4252-1922-4eb5-8c53-8ec5038d18a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:37.211886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:37.212910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:37.213787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:37.213787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:37.213787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:37.213787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:37.213787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:37.213787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:37.213787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:37.213787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:37.218009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 351/695] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:37.219527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b4252-1922-4eb5-8c53-8ec5038d18a0 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:37.276390 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caf426-9187-4871-b98e-e414d61e40fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:37.277488 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caf426-9187-4871-b98e-e414d61e40fd None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.277981 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caf426-9187-4871-b98e-e414d61e40fd None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.278465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caf426-9187-4871-b98e-e414d61e40fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.324527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17caf426-9187-4871-b98e-e414d61e40fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-17caf426-9187-4871-b98e-e414d61e40fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:37.330048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:37.332274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:37.334259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:37.334259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:37.334259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:37.334259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:37.334259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:37.334259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:37.334259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:37.334259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:37.334259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:37.334259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:37.334259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:37.334259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:37.338902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:37.339701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 345/696] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:37.344526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b4252-1922-4eb5-8c53-8ec5038d18a0 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:37.345433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caf426-9187-4871-b98e-e414d61e40fd None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:37.389547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b06c8-49c2-4cd7-bc5f-b94245f646e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:37.390448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b06c8-49c2-4cd7-bc5f-b94245f646e1 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.390733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b06c8-49c2-4cd7-bc5f-b94245f646e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.391006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b06c8-49c2-4cd7-bc5f-b94245f646e1 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.391868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b06c8-49c2-4cd7-bc5f-b94245f646e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:37.392769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b06c8-49c2-4cd7-bc5f-b94245f646e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:37.394477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b06c8-49c2-4cd7-bc5f-b94245f646e1 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:37.416019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 352/697] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:37.418299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b06c8-49c2-4cd7-bc5f-b94245f646e1 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:37.562376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b06c8-49c2-4cd7-bc5f-b94245f646e1 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:37.565442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:37.589449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caf426-9187-4871-b98e-e414d61e40fd None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:37.606431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c114523-2b47-48b6-867b-bfabf8b30032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:37.607223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c114523-2b47-48b6-867b-bfabf8b30032 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.607503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c114523-2b47-48b6-867b-bfabf8b30032 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.607780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c114523-2b47-48b6-867b-bfabf8b30032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.608519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c114523-2b47-48b6-867b-bfabf8b30032 None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=c5cf86ae159749cea0cc9783bffbfe75) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:37.608827 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c114523-2b47-48b6-867b-bfabf8b30032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:37.610633 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c114523-2b47-48b6-867b-bfabf8b30032 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:37.623285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 353/698] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:37 2022] GET /identity/v3/role_assignments?user.id=c5cf86ae159749cea0cc9783bffbfe75&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:37.626291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355fbf5-5dd6-4b0a-b75c-dd6801d134c3 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.627185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355fbf5-5dd6-4b0a-b75c-dd6801d134c3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.629449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355fbf5-5dd6-4b0a-b75c-dd6801d134c3 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.638219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 354/699] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:37 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:37.649884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6549c-538a-4368-b565-ea260db8dca9 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.651955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6549c-538a-4368-b565-ea260db8dca9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.653054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6549c-538a-4368-b565-ea260db8dca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.697172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cf0a5-ef8a-415c-89c8-3eabe57bc85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:37.697808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cf0a5-ef8a-415c-89c8-3eabe57bc85b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.698183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cf0a5-ef8a-415c-89c8-3eabe57bc85b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.698568 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cf0a5-ef8a-415c-89c8-3eabe57bc85b None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.699281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255cf0a5-ef8a-415c-89c8-3eabe57bc85b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:37.699590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255cf0a5-ef8a-415c-89c8-3eabe57bc85b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:37.700990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255cf0a5-ef8a-415c-89c8-3eabe57bc85b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:37.703470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-14d6549c-538a-4368-b565-ea260db8dca9 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:37.705972 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-14d6549c-538a-4368-b565-ea260db8dca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:37.730689 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 346/700] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:37.742083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:37.744447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cf0a5-ef8a-415c-89c8-3eabe57bc85b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:37.843383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 355/701] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:37.909841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cf0a5-ef8a-415c-89c8-3eabe57bc85b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:37.986079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d28102-0779-468e-8684-b9fcba8629d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:37.986747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d28102-0779-468e-8684-b9fcba8629d1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.987030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d28102-0779-468e-8684-b9fcba8629d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.987334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d28102-0779-468e-8684-b9fcba8629d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.007085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d28102-0779-468e-8684-b9fcba8629d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:38.008625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d28102-0779-468e-8684-b9fcba8629d1 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:38.014582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-c5d28102-0779-468e-8684-b9fcba8629d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:38.035242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:38.036299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:38.037230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:38.037230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:38.037230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:38.037230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 347/702] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:38.043023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6549c-538a-4368-b565-ea260db8dca9 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:38.194897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6549c-538a-4368-b565-ea260db8dca9 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:38.213541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d28102-0779-468e-8684-b9fcba8629d1 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:38.240072 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00932d-6319-43b7-8def-3da01a7cd9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:38.240707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00932d-6319-43b7-8def-3da01a7cd9d4 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.240987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00932d-6319-43b7-8def-3da01a7cd9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.241318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00932d-6319-43b7-8def-3da01a7cd9d4 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.242045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00932d-6319-43b7-8def-3da01a7cd9d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:38.242349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00932d-6319-43b7-8def-3da01a7cd9d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:38.245006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00932d-6319-43b7-8def-3da01a7cd9d4 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:38.275305 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 356/703] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:38.280176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00932d-6319-43b7-8def-3da01a7cd9d4 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:38.311677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:38.400696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d28102-0779-468e-8684-b9fcba8629d1 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:38.457981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00932d-6319-43b7-8def-3da01a7cd9d4 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:38.497986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b4ecd-6458-42e3-85e1-efc9e4f4b1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:38.498999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b4ecd-6458-42e3-85e1-efc9e4f4b1a1 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.499667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b4ecd-6458-42e3-85e1-efc9e4f4b1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.500098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b4ecd-6458-42e3-85e1-efc9e4f4b1a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.509048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b4ecd-6458-42e3-85e1-efc9e4f4b1a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:38.510808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b4ecd-6458-42e3-85e1-efc9e4f4b1a1 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:38.519931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde2a6a-4e6a-4b3c-b3b2-4823db6fc250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:38.520581 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2a6a-4e6a-4b3c-b3b2-4823db6fc250 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.520860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2a6a-4e6a-4b3c-b3b2-4823db6fc250 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.521131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2a6a-4e6a-4b3c-b3b2-4823db6fc250 None admin] PATH_INFO: `/v3/services/network` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.521722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde2a6a-4e6a-4b3c-b3b2-4823db6fc250 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:38.523137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde2a6a-4e6a-4b3c-b3b2-4823db6fc250 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-035b4ecd-6458-42e3-85e1-efc9e4f4b1a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:38.525597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:38.526627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:38.527794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:38.527794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:38.527794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:38.527794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:38.527794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:38.527794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:38.527794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:38.527794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:38.527794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:38.527794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:38.530012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 357/704] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 09:35:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:38.538106 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b4ecd-6458-42e3-85e1-efc9e4f4b1a1 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-ebde2a6a-4e6a-4b3c-b3b2-4823db6fc250 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:38.557381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:38.558566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:35:38.559437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:38.567181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 348/705] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:38 2022] GET /identity/v3/services/network => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:38.571523 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde2a6a-4e6a-4b3c-b3b2-4823db6fc250 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:38.704769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:38.715762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:38.753773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde2a6a-4e6a-4b3c-b3b2-4823db6fc250 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:38.761329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b4ecd-6458-42e3-85e1-efc9e4f4b1a1 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:38.796702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65e444-0db7-4b5c-8a58-02cd4b301be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:38.797478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65e444-0db7-4b5c-8a58-02cd4b301be2 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.797870 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65e444-0db7-4b5c-8a58-02cd4b301be2 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.798246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65e444-0db7-4b5c-8a58-02cd4b301be2 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.798996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65e444-0db7-4b5c-8a58-02cd4b301be2 None admin] RBAC: Adding query filter params (name=network) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:38.799429 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65e444-0db7-4b5c-8a58-02cd4b301be2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:38.801111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65e444-0db7-4b5c-8a58-02cd4b301be2 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:38.814148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 349/706] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:38.815951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b04667-c5c5-4f37-b82a-37bdec17b27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:38.816860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b04667-c5c5-4f37-b82a-37bdec17b27f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.817376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b04667-c5c5-4f37-b82a-37bdec17b27f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.817872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b04667-c5c5-4f37-b82a-37bdec17b27f None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.818826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b04667-c5c5-4f37-b82a-37bdec17b27f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:38.818935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323d4ba-c3b1-459e-8424-973adf0a965c None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.819396 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323d4ba-c3b1-459e-8424-973adf0a965c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.819680 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b04667-c5c5-4f37-b82a-37bdec17b27f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:38.819889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323d4ba-c3b1-459e-8424-973adf0a965c None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.821549 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b04667-c5c5-4f37-b82a-37bdec17b27f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:38.826759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 350/707] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:38.826759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323d4ba-c3b1-459e-8424-973adf0a965c None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:38.858517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 358/708] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 09:35:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:38.858517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eee3c8-4fe8-45bd-bd59-a9e41551f8ad None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.858517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eee3c8-4fe8-45bd-bd59-a9e41551f8ad None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.858517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eee3c8-4fe8-45bd-bd59-a9e41551f8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.878248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-45eee3c8-4fe8-45bd-bd59-a9e41551f8ad None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:38.879585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-45eee3c8-4fe8-45bd-bd59-a9e41551f8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:38.950829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323d4ba-c3b1-459e-8424-973adf0a965c None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:38.959581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 359/709] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:38.960681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eee3c8-4fe8-45bd-bd59-a9e41551f8ad None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:38.986997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:39.000275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c291156-2602-4f5f-af07-bc805406fdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:39.000909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c291156-2602-4f5f-af07-bc805406fdf7 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:39.001188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c291156-2602-4f5f-af07-bc805406fdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:39.001460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c291156-2602-4f5f-af07-bc805406fdf7 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:39.002129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c291156-2602-4f5f-af07-bc805406fdf7 None admin] RBAC: Adding query filter params (type=network) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:39.002433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c291156-2602-4f5f-af07-bc805406fdf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:39.004072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c291156-2602-4f5f-af07-bc805406fdf7 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:39.016440 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 351/710] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:38 2022] GET /identity/v3/services?type=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:39.016440 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175f7b3-816c-4d5d-b5ba-0fe37c4a73db None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:39.016440 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175f7b3-816c-4d5d-b5ba-0fe37c4a73db None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:39.016440 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175f7b3-816c-4d5d-b5ba-0fe37c4a73db None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:39.019588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 352/711] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:39.023163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a5c6e-e3ab-4c3e-9c1f-f98e00cbf68c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:39.023163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a5c6e-e3ab-4c3e-9c1f-f98e00cbf68c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:39.023163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a5c6e-e3ab-4c3e-9c1f-f98e00cbf68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:39.043009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-2a9a5c6e-e3ab-4c3e-9c1f-f98e00cbf68c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:39.044367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2a9a5c6e-e3ab-4c3e-9c1f-f98e00cbf68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:39.079140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:39.127850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 353/712] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:35:39.169591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eee3c8-4fe8-45bd-bd59-a9e41551f8ad None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:39.225194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11555118-60fd-4087-bc80-b904c638b0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:39.225818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11555118-60fd-4087-bc80-b904c638b0d9 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:39.226099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11555118-60fd-4087-bc80-b904c638b0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:39.226380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11555118-60fd-4087-bc80-b904c638b0d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:39.296183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a5c6e-e3ab-4c3e-9c1f-f98e00cbf68c None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:39.345612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11555118-60fd-4087-bc80-b904c638b0d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:39.348704 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11555118-60fd-4087-bc80-b904c638b0d9 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:39.439389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:39.477124 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a5c6e-e3ab-4c3e-9c1f-f98e00cbf68c None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:39.503008 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 360/713] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:39.508021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:39.508243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11555118-60fd-4087-bc80-b904c638b0d9 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:39.534783 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758871c-e5d9-4340-90fe-5a36ca126802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:39.535736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758871c-e5d9-4340-90fe-5a36ca126802 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:39.536188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758871c-e5d9-4340-90fe-5a36ca126802 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:39.536619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758871c-e5d9-4340-90fe-5a36ca126802 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:39.537976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758871c-e5d9-4340-90fe-5a36ca126802 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:39.539883 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758871c-e5d9-4340-90fe-5a36ca126802 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-5758871c-e5d9-4340-90fe-5a36ca126802 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:39.553868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:39.555199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:35:39.556363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:39.560151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 354/714] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Aug 31 09:35:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:39.566505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758871c-e5d9-4340-90fe-5a36ca126802 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:39.640514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11555118-60fd-4087-bc80-b904c638b0d9 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:39.679740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9832e-b008-4901-89e6-1995b8a8125f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:39.680754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9832e-b008-4901-89e6-1995b8a8125f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:39.681195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9832e-b008-4901-89e6-1995b8a8125f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:39.681685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9832e-b008-4901-89e6-1995b8a8125f None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:39.685420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758871c-e5d9-4340-90fe-5a36ca126802 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:39.693610 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9832e-b008-4901-89e6-1995b8a8125f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-e1f9832e-b008-4901-89e6-1995b8a8125f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:39.699229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:39.700358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:39.701254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:39.701254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:39.701254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:39.701254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:39.701254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:39.701254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:39.701254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:39.701254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:39.701254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:39.701254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:39.701254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:39.701254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:39.703925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 361/715] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:39.705035 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9832e-b008-4901-89e6-1995b8a8125f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:39.725193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da19dc-214c-420c-841e-6fa529c16a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:39.726005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da19dc-214c-420c-841e-6fa529c16a38 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:39.726356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da19dc-214c-420c-841e-6fa529c16a38 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:39.726731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da19dc-214c-420c-841e-6fa529c16a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:39.746377 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da19dc-214c-420c-841e-6fa529c16a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:39.746728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:39.749057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da19dc-214c-420c-841e-6fa529c16a38 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-d1da19dc-214c-420c-841e-6fa529c16a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:39.766080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:39.767171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:39.768083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:39.768083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:39.768083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:39.772248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 355/716] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:39.773411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da19dc-214c-420c-841e-6fa529c16a38 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:39.825391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:39.828075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9832e-b008-4901-89e6-1995b8a8125f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:39.864680 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22495c5-aafe-4c93-a499-61da4d8ef0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:39.865662 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22495c5-aafe-4c93-a499-61da4d8ef0b2 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:39.866149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22495c5-aafe-4c93-a499-61da4d8ef0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:39.866605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22495c5-aafe-4c93-a499-61da4d8ef0b2 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:39.867485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22495c5-aafe-4c93-a499-61da4d8ef0b2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:39.868019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22495c5-aafe-4c93-a499-61da4d8ef0b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:39.870360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22495c5-aafe-4c93-a499-61da4d8ef0b2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:39.880299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 362/717] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Aug 31 09:35:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:39.881576 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22495c5-aafe-4c93-a499-61da4d8ef0b2 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:39.903065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da19dc-214c-420c-841e-6fa529c16a38 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:39.941381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbc440-37ad-4e1c-a87e-38f793debee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:39.941987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbc440-37ad-4e1c-a87e-38f793debee9 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:39.942264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbc440-37ad-4e1c-a87e-38f793debee9 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:39.942537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbc440-37ad-4e1c-a87e-38f793debee9 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:39.943247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbc440-37ad-4e1c-a87e-38f793debee9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:39.943550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbc440-37ad-4e1c-a87e-38f793debee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:39.944942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbc440-37ad-4e1c-a87e-38f793debee9 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:39.955347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 356/718] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:39.956200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbc440-37ad-4e1c-a87e-38f793debee9 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:39.975854 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:39.993528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22495c5-aafe-4c93-a499-61da4d8ef0b2 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:40.033539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5c3ad-e962-4c50-b21f-62843e03b0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:40.034601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5c3ad-e962-4c50-b21f-62843e03b0ac None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.035175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5c3ad-e962-4c50-b21f-62843e03b0ac None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.035672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5c3ad-e962-4c50-b21f-62843e03b0ac None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.036546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5c3ad-e962-4c50-b21f-62843e03b0ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:40.037069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5c3ad-e962-4c50-b21f-62843e03b0ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:40.038672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5c3ad-e962-4c50-b21f-62843e03b0ac None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:40.052658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 363/719] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:40.053845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5c3ad-e962-4c50-b21f-62843e03b0ac None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:40.092121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:40.099803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbc440-37ad-4e1c-a87e-38f793debee9 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:40.140029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540f133-24b0-477c-8df6-c7871278b284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:40.140650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540f133-24b0-477c-8df6-c7871278b284 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.140928 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540f133-24b0-477c-8df6-c7871278b284 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.141199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540f133-24b0-477c-8df6-c7871278b284 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.141832 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540f133-24b0-477c-8df6-c7871278b284 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:40.142131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540f133-24b0-477c-8df6-c7871278b284 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:40.143668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540f133-24b0-477c-8df6-c7871278b284 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:40.155154 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 357/720] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Aug 31 09:35:39 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:40.155154 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540f133-24b0-477c-8df6-c7871278b284 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:40.190258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5c3ad-e962-4c50-b21f-62843e03b0ac None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:40.258239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3675b-3ae1-4099-ab38-59f162166d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:40.259319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3675b-3ae1-4099-ab38-59f162166d26 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.260402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3675b-3ae1-4099-ab38-59f162166d26 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.260894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3675b-3ae1-4099-ab38-59f162166d26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.270579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540f133-24b0-477c-8df6-c7871278b284 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:40.284481 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3675b-3ae1-4099-ab38-59f162166d26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:40.286324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3675b-3ae1-4099-ab38-59f162166d26 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:40.294156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-aba3675b-3ae1-4099-ab38-59f162166d26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:40.315558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:40.316584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:40.317458 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:40.317458 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:40.317458 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:40.319013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 364/721] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:40.319978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dd274-93b2-4243-b1ed-1bfef52a2713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:40.320904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dd274-93b2-4243-b1ed-1bfef52a2713 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.321338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dd274-93b2-4243-b1ed-1bfef52a2713 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.321780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dd274-93b2-4243-b1ed-1bfef52a2713 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.326351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3675b-3ae1-4099-ab38-59f162166d26 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:40.341215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dd274-93b2-4243-b1ed-1bfef52a2713 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:40.343003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dd274-93b2-4243-b1ed-1bfef52a2713 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-116dd274-93b2-4243-b1ed-1bfef52a2713 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:40.366851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:40.368079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:40.369812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:40.369812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:40.369812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:40.369812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:40.369812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:40.369812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:40.369812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:40.369812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:40.375831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 358/722] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:40.389044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dd274-93b2-4243-b1ed-1bfef52a2713 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:40.430472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:40.524824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3675b-3ae1-4099-ab38-59f162166d26 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:40.548381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dd274-93b2-4243-b1ed-1bfef52a2713 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:40.576902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e859b-f238-4ede-a438-20e3b3ae1d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:40.578503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e859b-f238-4ede-a438-20e3b3ae1d9b None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.579381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e859b-f238-4ede-a438-20e3b3ae1d9b None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.579917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e859b-f238-4ede-a438-20e3b3ae1d9b None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.581087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e859b-f238-4ede-a438-20e3b3ae1d9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:40.582025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e859b-f238-4ede-a438-20e3b3ae1d9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:40.584192 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e859b-f238-4ede-a438-20e3b3ae1d9b None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:40.595177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a8183-3176-4633-921d-551f706b1619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:40.595730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a8183-3176-4633-921d-551f706b1619 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.596013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a8183-3176-4633-921d-551f706b1619 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.596289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a8183-3176-4633-921d-551f706b1619 None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.596998 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a8183-3176-4633-921d-551f706b1619 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:40.597302 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a8183-3176-4633-921d-551f706b1619 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:40.599461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a8183-3176-4633-921d-551f706b1619 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:40.605093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 365/723] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:40.609313 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e859b-f238-4ede-a438-20e3b3ae1d9b None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:40.617867 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 359/724] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:40.628993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a8183-3176-4633-921d-551f706b1619 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:40.714420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:40.788446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:40.812877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a8183-3176-4633-921d-551f706b1619 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:40.818441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e859b-f238-4ede-a438-20e3b3ae1d9b None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:40.860897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f6ff4-f5f7-4f6f-bce5-16b3aa8bd908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:40.861442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f6ff4-f5f7-4f6f-bce5-16b3aa8bd908 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.861727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f6ff4-f5f7-4f6f-bce5-16b3aa8bd908 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.862001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f6ff4-f5f7-4f6f-bce5-16b3aa8bd908 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.862967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f6ff4-f5f7-4f6f-bce5-16b3aa8bd908 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=ef72bb95992c4a21ae2dbb2a08c51421) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:40.863307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f6ff4-f5f7-4f6f-bce5-16b3aa8bd908 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:40.864911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f6ff4-f5f7-4f6f-bce5-16b3aa8bd908 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:40.883174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 360/725] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:40 2022] GET /identity/v3/role_assignments?user.id=ef72bb95992c4a21ae2dbb2a08c51421&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:40.918299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef351b3-5bcb-483d-826c-81e904870528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:40.919239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef351b3-5bcb-483d-826c-81e904870528 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.919700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef351b3-5bcb-483d-826c-81e904870528 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.920127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef351b3-5bcb-483d-826c-81e904870528 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.936665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef351b3-5bcb-483d-826c-81e904870528 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:40.938397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef351b3-5bcb-483d-826c-81e904870528 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-2ef351b3-5bcb-483d-826c-81e904870528 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:40.955668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:40.956834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:40.957936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:40.957936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:40.957936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:40.957936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:40.957936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:40.957936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:40.957936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:40.957936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:40.957936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:40.957936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:40.965077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 366/726] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 09:35:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:40.967470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f6ff4-f5f7-4f6f-bce5-16b3aa8bd908 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:41.153243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f6ff4-f5f7-4f6f-bce5-16b3aa8bd908 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:41.173605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ea49c-a38b-475f-b742-ea204b2a8123 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.174026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ea49c-a38b-475f-b742-ea204b2a8123 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.174368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ea49c-a38b-475f-b742-ea204b2a8123 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.180410 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 367/727] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:41.182486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922529-581a-401d-b9a0-c50ed2efc33e None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.182776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922529-581a-401d-b9a0-c50ed2efc33e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.183049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922529-581a-401d-b9a0-c50ed2efc33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.217455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bf4ea-d18f-4ae6-bd32-736d15c5a25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:41.218332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bf4ea-d18f-4ae6-bd32-736d15c5a25d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.218780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bf4ea-d18f-4ae6-bd32-736d15c5a25d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.219219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bf4ea-d18f-4ae6-bd32-736d15c5a25d None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.220130 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bf4ea-d18f-4ae6-bd32-736d15c5a25d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:41.220592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bf4ea-d18f-4ae6-bd32-736d15c5a25d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:41.222258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bf4ea-d18f-4ae6-bd32-736d15c5a25d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:41.223451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-6b922529-581a-401d-b9a0-c50ed2efc33e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:41.226924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6b922529-581a-401d-b9a0-c50ed2efc33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:41.261117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 361/728] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Aug 31 09:35:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:41.268692 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bf4ea-d18f-4ae6-bd32-736d15c5a25d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:41.271885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:41.384723 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 368/729] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:41.386031 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2fd1e-0a03-44e1-a889-6aa4bf5435fb None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.386318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2fd1e-0a03-44e1-a889-6aa4bf5435fb None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.386591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2fd1e-0a03-44e1-a889-6aa4bf5435fb None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.392595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 369/730] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:41.394471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c50a1-8b47-4afa-a3bb-769067d355dc None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.394835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c50a1-8b47-4afa-a3bb-769067d355dc None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.395259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c50a1-8b47-4afa-a3bb-769067d355dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.434214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e53c50a1-8b47-4afa-a3bb-769067d355dc None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:41.435497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e53c50a1-8b47-4afa-a3bb-769067d355dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:41.456569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bf4ea-d18f-4ae6-bd32-736d15c5a25d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:41.528318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359f367-4aad-40cf-99f7-9af1d779b009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:41.532948 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359f367-4aad-40cf-99f7-9af1d779b009 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.533461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359f367-4aad-40cf-99f7-9af1d779b009 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.533930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359f367-4aad-40cf-99f7-9af1d779b009 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.542213 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:41.609899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 370/731] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:35:41.643196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359f367-4aad-40cf-99f7-9af1d779b009 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:41.645381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359f367-4aad-40cf-99f7-9af1d779b009 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:41.647675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:41.687163 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c50a1-8b47-4afa-a3bb-769067d355dc None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:41.798545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 362/732] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:41.805383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359f367-4aad-40cf-99f7-9af1d779b009 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:41.926328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c50a1-8b47-4afa-a3bb-769067d355dc None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.006313 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9778b3-5101-4fb8-be15-631be94a9952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.006928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9778b3-5101-4fb8-be15-631be94a9952 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.007251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9778b3-5101-4fb8-be15-631be94a9952 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.007529 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9778b3-5101-4fb8-be15-631be94a9952 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.032424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9778b3-5101-4fb8-be15-631be94a9952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:42.036340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359f367-4aad-40cf-99f7-9af1d779b009 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.038748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-ef9778b3-5101-4fb8-be15-631be94a9952 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:42.039011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:42.039964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:42.040834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 371/733] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:42.042227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9778b3-5101-4fb8-be15-631be94a9952 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.088959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:42.112059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73de25f-e724-4a31-9e04-20d741c6d31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.112957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73de25f-e724-4a31-9e04-20d741c6d31f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.113417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73de25f-e724-4a31-9e04-20d741c6d31f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.113878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73de25f-e724-4a31-9e04-20d741c6d31f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.128723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73de25f-e724-4a31-9e04-20d741c6d31f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.130298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73de25f-e724-4a31-9e04-20d741c6d31f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-a73de25f-e724-4a31-9e04-20d741c6d31f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:42.147958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:42.149189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:42.150785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:42.150785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:42.150785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:42.154567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 363/734] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:42.155520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73de25f-e724-4a31-9e04-20d741c6d31f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.179943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9778b3-5101-4fb8-be15-631be94a9952 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.220783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0451a3-4864-410c-a66b-663c92714151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.221944 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0451a3-4864-410c-a66b-663c92714151 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.222441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0451a3-4864-410c-a66b-663c92714151 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.222947 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0451a3-4864-410c-a66b-663c92714151 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.223954 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0451a3-4864-410c-a66b-663c92714151 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.226419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0451a3-4864-410c-a66b-663c92714151 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.245525 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 372/735] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:35:42 2022] POST /identity/v3/services => generated 240 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:42.246801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0451a3-4864-410c-a66b-663c92714151 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.345645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73de25f-e724-4a31-9e04-20d741c6d31f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.404924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:42.424586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0451a3-4864-410c-a66b-663c92714151 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.436820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23531e7-d8c4-4aca-8e9e-5c1e12616e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.437445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23531e7-d8c4-4aca-8e9e-5c1e12616e25 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.437725 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23531e7-d8c4-4aca-8e9e-5c1e12616e25 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.438029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23531e7-d8c4-4aca-8e9e-5c1e12616e25 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.439115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23531e7-d8c4-4aca-8e9e-5c1e12616e25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:42.441166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23531e7-d8c4-4aca-8e9e-5c1e12616e25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.443145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23531e7-d8c4-4aca-8e9e-5c1e12616e25 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.453477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 364/736] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:42.465579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23531e7-d8c4-4aca-8e9e-5c1e12616e25 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.488295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:42.499813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cab6da-1f8c-471c-a31c-b9de4062f7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.500933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cab6da-1f8c-471c-a31c-b9de4062f7c8 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.501533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cab6da-1f8c-471c-a31c-b9de4062f7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.502244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cab6da-1f8c-471c-a31c-b9de4062f7c8 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.503473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cab6da-1f8c-471c-a31c-b9de4062f7c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:42.504062 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cab6da-1f8c-471c-a31c-b9de4062f7c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.506177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cab6da-1f8c-471c-a31c-b9de4062f7c8 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.524615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 373/737] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:42.538040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cab6da-1f8c-471c-a31c-b9de4062f7c8 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.590009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23531e7-d8c4-4aca-8e9e-5c1e12616e25 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.628911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16966cb-ceba-4e74-afa9-004ac86f8912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.629411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16966cb-ceba-4e74-afa9-004ac86f8912 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.629709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16966cb-ceba-4e74-afa9-004ac86f8912 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.629990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16966cb-ceba-4e74-afa9-004ac86f8912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.642985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16966cb-ceba-4e74-afa9-004ac86f8912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.644685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16966cb-ceba-4e74-afa9-004ac86f8912 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-a16966cb-ceba-4e74-afa9-004ac86f8912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:42.661293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:42.662678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:42.663970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:42.663970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:42.663970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:42.667274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 365/738] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:42.669500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16966cb-ceba-4e74-afa9-004ac86f8912 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.707001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cab6da-1f8c-471c-a31c-b9de4062f7c8 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.746822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:42.748714 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47810b8-9197-4a06-84dd-d47e395ff95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.749754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47810b8-9197-4a06-84dd-d47e395ff95b None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.750395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47810b8-9197-4a06-84dd-d47e395ff95b None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.750859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47810b8-9197-4a06-84dd-d47e395ff95b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.758926 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47810b8-9197-4a06-84dd-d47e395ff95b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.760856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47810b8-9197-4a06-84dd-d47e395ff95b None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-e47810b8-9197-4a06-84dd-d47e395ff95b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:42.773976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:42.775471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:42.776395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:42.776395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:42.776395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:42.776395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:42.776395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:42.776395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:42.776395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:42.776395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:42.778896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 374/739] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:42.787173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47810b8-9197-4a06-84dd-d47e395ff95b None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.809825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16966cb-ceba-4e74-afa9-004ac86f8912 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.823976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:42.890738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9c29d-028d-4ada-a9d5-74c1e27d060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.891366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9c29d-028d-4ada-a9d5-74c1e27d060b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.891644 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9c29d-028d-4ada-a9d5-74c1e27d060b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.891917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9c29d-028d-4ada-a9d5-74c1e27d060b None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.892626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9c29d-028d-4ada-a9d5-74c1e27d060b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:42.892931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9c29d-028d-4ada-a9d5-74c1e27d060b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.894327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9c29d-028d-4ada-a9d5-74c1e27d060b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.925414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 366/740] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:42.932072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9c29d-028d-4ada-a9d5-74c1e27d060b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.959881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47810b8-9197-4a06-84dd-d47e395ff95b None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:43.000036 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93575b-f27e-4ab6-b06f-5f07fc23dd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:43.000669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93575b-f27e-4ab6-b06f-5f07fc23dd81 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.000948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93575b-f27e-4ab6-b06f-5f07fc23dd81 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.001221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93575b-f27e-4ab6-b06f-5f07fc23dd81 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.001927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93575b-f27e-4ab6-b06f-5f07fc23dd81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:43.002233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93575b-f27e-4ab6-b06f-5f07fc23dd81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:43.003715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93575b-f27e-4ab6-b06f-5f07fc23dd81 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:43.019257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 375/741] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:43.026130 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93575b-f27e-4ab6-b06f-5f07fc23dd81 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:43.138040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9c29d-028d-4ada-a9d5-74c1e27d060b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:43.180486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:43.184078 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:43.195643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b359a-36b7-4f20-9786-ecbafa0e01bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:43.196560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b359a-36b7-4f20-9786-ecbafa0e01bc None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.197528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b359a-36b7-4f20-9786-ecbafa0e01bc None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.197805 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b359a-36b7-4f20-9786-ecbafa0e01bc None admin] PATH_INFO: `/v3/users/swift` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.208433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b359a-36b7-4f20-9786-ecbafa0e01bc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:43.209905 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b359a-36b7-4f20-9786-ecbafa0e01bc None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:43.210323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93575b-f27e-4ab6-b06f-5f07fc23dd81 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-0f9b359a-36b7-4f20-9786-ecbafa0e01bc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:43.225768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:43.227528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:43.228504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:43.228504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:43.228504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:43.228504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:43.228504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:43.228504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:43.228504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:43.228504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:43.228504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:43.228504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:43.228504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 367/742] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:43.229100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a467a26-3b50-4f01-a498-d51a31b03ae3 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.229100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a467a26-3b50-4f01-a498-d51a31b03ae3 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.229890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a467a26-3b50-4f01-a498-d51a31b03ae3 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.230874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 368/743] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:43.235983 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a467a26-3b50-4f01-a498-d51a31b03ae3 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:43.270070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb2cef-3795-4894-adc5-d0590ef9baa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:43.270863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb2cef-3795-4894-adc5-d0590ef9baa2 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.271319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb2cef-3795-4894-adc5-d0590ef9baa2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.271690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb2cef-3795-4894-adc5-d0590ef9baa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.272539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb2cef-3795-4894-adc5-d0590ef9baa2 None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=0a2e5c1014f1441aa47d5de46462b6d9) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:43.272938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb2cef-3795-4894-adc5-d0590ef9baa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:43.274501 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb2cef-3795-4894-adc5-d0590ef9baa2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:43.286380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 376/744] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:43 2022] GET /identity/v3/role_assignments?user.id=0a2e5c1014f1441aa47d5de46462b6d9&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:43.286380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d663514-e8ec-4823-866e-6935460108f7 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.286380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d663514-e8ec-4823-866e-6935460108f7 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.286380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d663514-e8ec-4823-866e-6935460108f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.304718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-8d663514-e8ec-4823-866e-6935460108f7 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:43.306156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-8d663514-e8ec-4823-866e-6935460108f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:43.415391 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 377/745] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:43 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:43.417552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c7278-3c24-4561-8b58-244a9630fa43 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.417926 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c7278-3c24-4561-8b58-244a9630fa43 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.418205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c7278-3c24-4561-8b58-244a9630fa43 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.423896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 378/746] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:43.425920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c52152-f3dc-41c8-aef7-da519ef97740 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.426209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c52152-f3dc-41c8-aef7-da519ef97740 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.426486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c52152-f3dc-41c8-aef7-da519ef97740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.456504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-06c52152-f3dc-41c8-aef7-da519ef97740 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:43.458365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-06c52152-f3dc-41c8-aef7-da519ef97740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:43.474532 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a467a26-3b50-4f01-a498-d51a31b03ae3 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:43.533129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc7c46-79e0-410d-8515-903d6276b7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:43.534592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc7c46-79e0-410d-8515-903d6276b7fd None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.535151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc7c46-79e0-410d-8515-903d6276b7fd None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.535671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc7c46-79e0-410d-8515-903d6276b7fd None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.537198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc7c46-79e0-410d-8515-903d6276b7fd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:43.537921 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc7c46-79e0-410d-8515-903d6276b7fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:43.540083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc7c46-79e0-410d-8515-903d6276b7fd None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:43.545740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:43.564017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 379/747] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:43 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:35:43.578629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 369/748] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Aug 31 09:35:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:43.580570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c52152-f3dc-41c8-aef7-da519ef97740 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:43.693752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c52152-f3dc-41c8-aef7-da519ef97740 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:43.738646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a908bf-41a9-4bc5-828a-d4575043e3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:43.740210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a908bf-41a9-4bc5-828a-d4575043e3df None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.741032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a908bf-41a9-4bc5-828a-d4575043e3df None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.741798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a908bf-41a9-4bc5-828a-d4575043e3df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.859528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:43.865769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a908bf-41a9-4bc5-828a-d4575043e3df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:43.872831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a908bf-41a9-4bc5-828a-d4575043e3df None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:43.888251 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc7c46-79e0-410d-8515-903d6276b7fd None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:43.941283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:44.007355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc7c46-79e0-410d-8515-903d6276b7fd None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:44.007530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 380/749] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:43 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:35:44.008863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a908bf-41a9-4bc5-828a-d4575043e3df None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:44.050621 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5aaf3a-220e-401b-8f50-54c27b821feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:44.051656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aaf3a-220e-401b-8f50-54c27b821feb None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:44.052144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aaf3a-220e-401b-8f50-54c27b821feb None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:44.052611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aaf3a-220e-401b-8f50-54c27b821feb None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:44.053535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5aaf3a-220e-401b-8f50-54c27b821feb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:44.055355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5aaf3a-220e-401b-8f50-54c27b821feb None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:44.111627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 370/750] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:35:43 2022] POST /identity/v3/services => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:44.120346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:44.129222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a908bf-41a9-4bc5-828a-d4575043e3df None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:44.162860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5aaf3a-220e-401b-8f50-54c27b821feb None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:44.178213 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142e93c-ab79-4d6a-8810-ba55728bd5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:44.178892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142e93c-ab79-4d6a-8810-ba55728bd5b5 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:44.179285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142e93c-ab79-4d6a-8810-ba55728bd5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:44.179564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142e93c-ab79-4d6a-8810-ba55728bd5b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:44.195162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142e93c-ab79-4d6a-8810-ba55728bd5b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:44.195162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142e93c-ab79-4d6a-8810-ba55728bd5b5 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-4142e93c-ab79-4d6a-8810-ba55728bd5b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:44.225237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:44.226616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:44.228749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:44.228749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:44.228749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:44.236573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 381/751] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:44.240670 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142e93c-ab79-4d6a-8810-ba55728bd5b5 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:44.362167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5aaf3a-220e-401b-8f50-54c27b821feb None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:44.369891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:44.396129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142e93c-ab79-4d6a-8810-ba55728bd5b5 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:44.403745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3f973-71b2-4413-ab13-e0a5dc2ea5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:44.404503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3f973-71b2-4413-ab13-e0a5dc2ea5e6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:44.404873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3f973-71b2-4413-ab13-e0a5dc2ea5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:44.405283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3f973-71b2-4413-ab13-e0a5dc2ea5e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:44.417767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3f973-71b2-4413-ab13-e0a5dc2ea5e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-dcc3f973-71b2-4413-ab13-e0a5dc2ea5e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:44.422893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:44.424067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:44.425066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 371/752] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:44.430806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3f973-71b2-4413-ab13-e0a5dc2ea5e6 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:44.439945 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6a3f0-70c7-487f-b077-039ad828ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:44.440822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6a3f0-70c7-487f-b077-039ad828ad02 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:44.441253 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6a3f0-70c7-487f-b077-039ad828ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:44.441660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6a3f0-70c7-487f-b077-039ad828ad02 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:44.442505 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6a3f0-70c7-487f-b077-039ad828ad02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:44.442952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6a3f0-70c7-487f-b077-039ad828ad02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:44.444584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6a3f0-70c7-487f-b077-039ad828ad02 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:44.462161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 382/753] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:44.465480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6a3f0-70c7-487f-b077-039ad828ad02 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:44.469543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:44.584822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3f973-71b2-4413-ab13-e0a5dc2ea5e6 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:44.632744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6a3f0-70c7-487f-b077-039ad828ad02 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:44.638974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744d76d-a1a1-4615-98b9-ee7644412482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:44.640189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744d76d-a1a1-4615-98b9-ee7644412482 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:44.640668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744d76d-a1a1-4615-98b9-ee7644412482 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:44.641139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744d76d-a1a1-4615-98b9-ee7644412482 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:44.642007 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744d76d-a1a1-4615-98b9-ee7644412482 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:44.642548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744d76d-a1a1-4615-98b9-ee7644412482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:44.644714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744d76d-a1a1-4615-98b9-ee7644412482 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:44.657799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 372/754] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:44.662917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744d76d-a1a1-4615-98b9-ee7644412482 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:44.720070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:44.727852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8dd9a-fe54-4f8e-80ad-107816291e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:44.729012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8dd9a-fe54-4f8e-80ad-107816291e11 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:44.729012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8dd9a-fe54-4f8e-80ad-107816291e11 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:44.729455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8dd9a-fe54-4f8e-80ad-107816291e11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:44.744913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8dd9a-fe54-4f8e-80ad-107816291e11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:44.745990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8dd9a-fe54-4f8e-80ad-107816291e11 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-4ca8dd9a-fe54-4f8e-80ad-107816291e11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:44.768575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:44.769504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:44.770376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:44.770376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:44.770376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:44.770376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:44.770376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:44.770376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:44.770376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:44.770376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:44.776559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 383/755] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:44.778252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54140712-23d0-4a5f-8323-a036b7f501c6 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:44.778754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54140712-23d0-4a5f-8323-a036b7f501c6 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:44.779256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54140712-23d0-4a5f-8323-a036b7f501c6 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:44.783562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 384/756] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:44.785159 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54140712-23d0-4a5f-8323-a036b7f501c6 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:44.819888 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744d76d-a1a1-4615-98b9-ee7644412482 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:44.863535 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:44.869411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5922d79-6600-40f9-bac4-9b6fcbc36534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:44.870058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5922d79-6600-40f9-bac4-9b6fcbc36534 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:44.870333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5922d79-6600-40f9-bac4-9b6fcbc36534 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:44.870623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5922d79-6600-40f9-bac4-9b6fcbc36534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:44.884150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5922d79-6600-40f9-bac4-9b6fcbc36534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:44.887096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5922d79-6600-40f9-bac4-9b6fcbc36534 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-d5922d79-6600-40f9-bac4-9b6fcbc36534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:44.904801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:44.906283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:44.907178 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:44.907178 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:44.907178 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:44.914218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 373/757] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:44.914218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d78825-45e3-42d6-992a-42ab8871d050 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:44.914218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d78825-45e3-42d6-992a-42ab8871d050 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:44.914218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d78825-45e3-42d6-992a-42ab8871d050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:44.926392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54140712-23d0-4a5f-8323-a036b7f501c6 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:44.940836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-27d78825-45e3-42d6-992a-42ab8871d050 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:44.942253 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-27d78825-45e3-42d6-992a-42ab8871d050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:44.981712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ecb16-10bc-4ff1-afd3-85895834714d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:44.983670 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ecb16-10bc-4ff1-afd3-85895834714d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:44.983950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ecb16-10bc-4ff1-afd3-85895834714d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:44.984210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ecb16-10bc-4ff1-afd3-85895834714d None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:44.984913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ecb16-10bc-4ff1-afd3-85895834714d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:44.985208 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ecb16-10bc-4ff1-afd3-85895834714d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:44.988208 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ecb16-10bc-4ff1-afd3-85895834714d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:45.005840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 385/758] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:45.007386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ecb16-10bc-4ff1-afd3-85895834714d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:45.027160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 374/759] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:45.027160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d78825-45e3-42d6-992a-42ab8871d050 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:45.047522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:45.210330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ecb16-10bc-4ff1-afd3-85895834714d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:45.223311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d78825-45e3-42d6-992a-42ab8871d050 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:45.257990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:45.279198 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca495ad4-1a9c-45da-9390-ba45fc913138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:45.279815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca495ad4-1a9c-45da-9390-ba45fc913138 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:45.280091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca495ad4-1a9c-45da-9390-ba45fc913138 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:45.280360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca495ad4-1a9c-45da-9390-ba45fc913138 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:45.281011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca495ad4-1a9c-45da-9390-ba45fc913138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:45.281307 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca495ad4-1a9c-45da-9390-ba45fc913138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:45.283251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca495ad4-1a9c-45da-9390-ba45fc913138 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:45.288000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070314f0-140b-4637-a277-13fcb6e248a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:45.288609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070314f0-140b-4637-a277-13fcb6e248a5 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:45.288880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070314f0-140b-4637-a277-13fcb6e248a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:45.289145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070314f0-140b-4637-a277-13fcb6e248a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:45.289864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070314f0-140b-4637-a277-13fcb6e248a5 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=c5cf86ae159749cea0cc9783bffbfe75) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:45.290222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070314f0-140b-4637-a277-13fcb6e248a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:45.293677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070314f0-140b-4637-a277-13fcb6e248a5 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:45.306547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 375/760] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:45 2022] GET /identity/v3/role_assignments?user.id=c5cf86ae159749cea0cc9783bffbfe75&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:45.309146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 386/761] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:45.316633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070314f0-140b-4637-a277-13fcb6e248a5 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:45.398697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca495ad4-1a9c-45da-9390-ba45fc913138 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:45.445287 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:45.494819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070314f0-140b-4637-a277-13fcb6e248a5 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:45.539220 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8237a5e-2fbc-48d2-9e48-174514f5bffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:45.539859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8237a5e-2fbc-48d2-9e48-174514f5bffd None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:45.540137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8237a5e-2fbc-48d2-9e48-174514f5bffd None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:45.540407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8237a5e-2fbc-48d2-9e48-174514f5bffd None admin] PATH_INFO: `/v3/users/nova` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:45.550471 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8237a5e-2fbc-48d2-9e48-174514f5bffd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:45.552119 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8237a5e-2fbc-48d2-9e48-174514f5bffd None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:45.567860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca495ad4-1a9c-45da-9390-ba45fc913138 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-b8237a5e-2fbc-48d2-9e48-174514f5bffd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:45.581852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:45.582904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:45.583894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:45.583894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:45.583894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:45.583894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:45.583894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:45.583894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:45.583894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:45.583894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:45.583894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:45.583894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:45.587280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 376/762] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 09:35:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:45.590356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8237a5e-2fbc-48d2-9e48-174514f5bffd None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:45.624114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a033fab-0094-41f4-808a-ea7a396b3806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:45.624776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a033fab-0094-41f4-808a-ea7a396b3806 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:45.625058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a033fab-0094-41f4-808a-ea7a396b3806 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:45.625333 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a033fab-0094-41f4-808a-ea7a396b3806 None admin] PATH_INFO: `/v3/services/network` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:45.625952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a033fab-0094-41f4-808a-ea7a396b3806 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:45.629512 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a033fab-0094-41f4-808a-ea7a396b3806 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-4a033fab-0094-41f4-808a-ea7a396b3806 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:45.657416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:45.658475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:35:45.659348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:45.661135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 387/763] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:45 2022] GET /identity/v3/services/network => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:45.667539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a033fab-0094-41f4-808a-ea7a396b3806 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:45.761785 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:45.801507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8237a5e-2fbc-48d2-9e48-174514f5bffd None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:45.854815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a033fab-0094-41f4-808a-ea7a396b3806 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:45.872857 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac105400-1665-403e-9242-189b1923a504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:45.873676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac105400-1665-403e-9242-189b1923a504 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:45.874055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac105400-1665-403e-9242-189b1923a504 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:45.874433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac105400-1665-403e-9242-189b1923a504 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:45.875446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac105400-1665-403e-9242-189b1923a504 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:45.875989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac105400-1665-403e-9242-189b1923a504 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:45.877810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac105400-1665-403e-9242-189b1923a504 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:45.880013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:45.903837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 377/764] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 09:35:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:45.903837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada1128-9402-4e4b-8e30-a5eeeabd5666 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:45.903837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada1128-9402-4e4b-8e30-a5eeeabd5666 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:45.903837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada1128-9402-4e4b-8e30-a5eeeabd5666 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:45.904718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02404dc4-39f9-4b22-a0b1-c8b4a9c04731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:45.905693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02404dc4-39f9-4b22-a0b1-c8b4a9c04731 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:45.906102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02404dc4-39f9-4b22-a0b1-c8b4a9c04731 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:45.906499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02404dc4-39f9-4b22-a0b1-c8b4a9c04731 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:45.907314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02404dc4-39f9-4b22-a0b1-c8b4a9c04731 None admin] RBAC: Adding query filter params (name=network) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:45.908953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 378/765] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:45.909756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02404dc4-39f9-4b22-a0b1-c8b4a9c04731 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:45.910213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f491e2-c0d1-46a6-b990-aa2e9acd6cb8 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:45.910607 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f491e2-c0d1-46a6-b990-aa2e9acd6cb8 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:45.910987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f491e2-c0d1-46a6-b990-aa2e9acd6cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:45.911876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02404dc4-39f9-4b22-a0b1-c8b4a9c04731 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:45.917991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 388/766] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:45.924090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02404dc4-39f9-4b22-a0b1-c8b4a9c04731 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:45.940308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-90f491e2-c0d1-46a6-b990-aa2e9acd6cb8 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:45.942098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-90f491e2-c0d1-46a6-b990-aa2e9acd6cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:46.030060 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 379/767] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:46.031410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f491e2-c0d1-46a6-b990-aa2e9acd6cb8 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.063941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02404dc4-39f9-4b22-a0b1-c8b4a9c04731 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.081374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:46.138650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0fc57-cdc9-4161-81b9-f0e7dbbedff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.140556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0fc57-cdc9-4161-81b9-f0e7dbbedff0 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.141048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0fc57-cdc9-4161-81b9-f0e7dbbedff0 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.141543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0fc57-cdc9-4161-81b9-f0e7dbbedff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.177291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0fc57-cdc9-4161-81b9-f0e7dbbedff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:46.179185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0fc57-cdc9-4161-81b9-f0e7dbbedff0 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-cdd0fc57-cdc9-4161-81b9-f0e7dbbedff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:46.205518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:46.206690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:46.207801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:46.207801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:46.207801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:46.207801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 389/768] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:46.212681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0fc57-cdc9-4161-81b9-f0e7dbbedff0 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.281205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f491e2-c0d1-46a6-b990-aa2e9acd6cb8 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.356396 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:46.386470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0fc57-cdc9-4161-81b9-f0e7dbbedff0 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.389216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcdebd-b36e-4794-a331-bcbe78fd05cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.390096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcdebd-b36e-4794-a331-bcbe78fd05cb None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.390370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcdebd-b36e-4794-a331-bcbe78fd05cb None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.390640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcdebd-b36e-4794-a331-bcbe78fd05cb None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.391303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcdebd-b36e-4794-a331-bcbe78fd05cb None admin] RBAC: Adding query filter params (type=network) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:46.391603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcdebd-b36e-4794-a331-bcbe78fd05cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:46.392977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcdebd-b36e-4794-a331-bcbe78fd05cb None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:46.404134 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 380/769] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:46.413202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcdebd-b36e-4794-a331-bcbe78fd05cb None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.446659 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cc936-5ab9-4c2c-a098-030e3b549f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.447832 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cc936-5ab9-4c2c-a098-030e3b549f57 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.448105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cc936-5ab9-4c2c-a098-030e3b549f57 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.448385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cc936-5ab9-4c2c-a098-030e3b549f57 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.449951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cc936-5ab9-4c2c-a098-030e3b549f57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:46.450263 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cc936-5ab9-4c2c-a098-030e3b549f57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:46.451754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cc936-5ab9-4c2c-a098-030e3b549f57 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:46.469784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 390/770] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:46.475402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cc936-5ab9-4c2c-a098-030e3b549f57 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.516944 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:46.547019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcdebd-b36e-4794-a331-bcbe78fd05cb None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.589023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546b1ed-7647-4891-bfb5-82cd82101bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.589623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546b1ed-7647-4891-bfb5-82cd82101bbf None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.589931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546b1ed-7647-4891-bfb5-82cd82101bbf None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.590214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546b1ed-7647-4891-bfb5-82cd82101bbf None admin] PATH_INFO: `/v3/endpoints` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.590875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546b1ed-7647-4891-bfb5-82cd82101bbf None admin] RBAC: Adding query filter params (interface=public, service_id=0c5592a136504cf58ee61d1cf39b2517, region_id=RegionOne) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:46.591204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546b1ed-7647-4891-bfb5-82cd82101bbf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:46.592601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546b1ed-7647-4891-bfb5-82cd82101bbf None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:46.603560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 381/771] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 09:35:46 2022] GET /identity/v3/endpoints?service_id=0c5592a136504cf58ee61d1cf39b2517&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:46.605063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546b1ed-7647-4891-bfb5-82cd82101bbf None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.648666 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cc936-5ab9-4c2c-a098-030e3b549f57 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.672490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:46.693319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9045426-b66a-4ccd-8da1-b516597f075a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.693978 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9045426-b66a-4ccd-8da1-b516597f075a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.694265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9045426-b66a-4ccd-8da1-b516597f075a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.694542 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9045426-b66a-4ccd-8da1-b516597f075a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.706754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9045426-b66a-4ccd-8da1-b516597f075a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-b9045426-b66a-4ccd-8da1-b516597f075a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:46.711771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:46.715429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 391/772] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:46.717212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9c9bf-1d24-4216-961d-703ad249aaf0 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.718534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9c9bf-1d24-4216-961d-703ad249aaf0 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.718534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9c9bf-1d24-4216-961d-703ad249aaf0 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.721569 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 392/773] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:46.722325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9c9bf-1d24-4216-961d-703ad249aaf0 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.756128 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546b1ed-7647-4891-bfb5-82cd82101bbf None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.805550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703f488-29ee-49af-9ebc-0eae929a0562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.806488 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703f488-29ee-49af-9ebc-0eae929a0562 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.806488 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703f488-29ee-49af-9ebc-0eae929a0562 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.806893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703f488-29ee-49af-9ebc-0eae929a0562 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.815294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703f488-29ee-49af-9ebc-0eae929a0562 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:46.816795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703f488-29ee-49af-9ebc-0eae929a0562 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-e703f488-29ee-49af-9ebc-0eae929a0562 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:46.833306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:46.834283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:46.835445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:46.835445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:46.835445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:46.835445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:46.835445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:46.835445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:46.835445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:46.835445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:46.835445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 382/774] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:46.835445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703f488-29ee-49af-9ebc-0eae929a0562 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.841563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:46.861971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9c9bf-1d24-4216-961d-703ad249aaf0 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.904947 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96010e-dcc7-4d31-bb30-ce15d7a18a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.905558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96010e-dcc7-4d31-bb30-ce15d7a18a0a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.905838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96010e-dcc7-4d31-bb30-ce15d7a18a0a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.906106 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96010e-dcc7-4d31-bb30-ce15d7a18a0a None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.906755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96010e-dcc7-4d31-bb30-ce15d7a18a0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:46.908229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96010e-dcc7-4d31-bb30-ce15d7a18a0a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:46.914705 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 393/775] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 09:35:46 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:46.916762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39254b67-3297-4242-b618-8a697b424168 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.917062 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39254b67-3297-4242-b618-8a697b424168 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.917250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39254b67-3297-4242-b618-8a697b424168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.942518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-39254b67-3297-4242-b618-8a697b424168 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:46.944707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-39254b67-3297-4242-b618-8a697b424168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:46.997572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703f488-29ee-49af-9ebc-0eae929a0562 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:47.020095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:47.041416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 394/776] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:47.042785 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39254b67-3297-4242-b618-8a697b424168 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.087165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a7b1b-2efd-4c81-b32d-a5c9323f98e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:47.089284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a7b1b-2efd-4c81-b32d-a5c9323f98e2 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.090058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a7b1b-2efd-4c81-b32d-a5c9323f98e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.090747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a7b1b-2efd-4c81-b32d-a5c9323f98e2 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.093204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a7b1b-2efd-4c81-b32d-a5c9323f98e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:47.093989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a7b1b-2efd-4c81-b32d-a5c9323f98e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:47.096653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a7b1b-2efd-4c81-b32d-a5c9323f98e2 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:47.121439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 383/777] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:47.128853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a7b1b-2efd-4c81-b32d-a5c9323f98e2 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.172415 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:47.205403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39254b67-3297-4242-b618-8a697b424168 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:47.246454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdba55-4a8e-4c46-b45b-ad265a7f7f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:47.247075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdba55-4a8e-4c46-b45b-ad265a7f7f30 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.247388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdba55-4a8e-4c46-b45b-ad265a7f7f30 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.247661 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdba55-4a8e-4c46-b45b-ad265a7f7f30 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.248374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdba55-4a8e-4c46-b45b-ad265a7f7f30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:47.248674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdba55-4a8e-4c46-b45b-ad265a7f7f30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:47.250154 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdba55-4a8e-4c46-b45b-ad265a7f7f30 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:47.264293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 395/778] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:47.270428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdba55-4a8e-4c46-b45b-ad265a7f7f30 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.334102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a7b1b-2efd-4c81-b32d-a5c9323f98e2 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:47.410158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8ada3-c562-4719-a99d-b5918cff9afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:47.410799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8ada3-c562-4719-a99d-b5918cff9afa None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.411075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8ada3-c562-4719-a99d-b5918cff9afa None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.411364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8ada3-c562-4719-a99d-b5918cff9afa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.429662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8ada3-c562-4719-a99d-b5918cff9afa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:47.433036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8ada3-c562-4719-a99d-b5918cff9afa None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:47.441394 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdba55-4a8e-4c46-b45b-ad265a7f7f30 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-01b8ada3-c562-4719-a99d-b5918cff9afa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:47.462219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:47.464609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:47.466400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:47.466400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:47.466400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:47.473769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 384/779] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:47.473769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8ada3-c562-4719-a99d-b5918cff9afa None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.488103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5dd1f-1465-489d-ba56-15f7cfe5cabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:47.488779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5dd1f-1465-489d-ba56-15f7cfe5cabb None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.489157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5dd1f-1465-489d-ba56-15f7cfe5cabb None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.489469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5dd1f-1465-489d-ba56-15f7cfe5cabb None admin] PATH_INFO: `/v3/projects/8ef8b586382941408cd88ee654c42d32/users/ef72bb95992c4a21ae2dbb2a08c51421/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.493099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b5dd1f-1465-489d-ba56-15f7cfe5cabb None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef8b586382941408cd88ee654c42d32, user_id=ef72bb95992c4a21ae2dbb2a08c51421, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:47.495354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b5dd1f-1465-489d-ba56-15f7cfe5cabb None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:47.507463 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:47.516161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 396/780] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:47 2022] PUT /identity/v3/projects/8ef8b586382941408cd88ee654c42d32/users/ef72bb95992c4a21ae2dbb2a08c51421/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:35:47.517827 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:47.519749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5dd1f-1465-489d-ba56-15f7cfe5cabb None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.626416 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8ada3-c562-4719-a99d-b5918cff9afa None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:47.665902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5dd1f-1465-489d-ba56-15f7cfe5cabb None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:47.675506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123758a-2480-4cef-a9a3-c44a8de5c5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:47.676419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123758a-2480-4cef-a9a3-c44a8de5c5d6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.676910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123758a-2480-4cef-a9a3-c44a8de5c5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.677359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123758a-2480-4cef-a9a3-c44a8de5c5d6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.678224 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123758a-2480-4cef-a9a3-c44a8de5c5d6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:47.680024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123758a-2480-4cef-a9a3-c44a8de5c5d6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-b123758a-2480-4cef-a9a3-c44a8de5c5d6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:47.691793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:47.692950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:35:47.693880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:47.697359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 385/781] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Aug 31 09:35:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:47.704044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d5e18-9d79-40ac-9561-fd0af0285e83 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.704441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d5e18-9d79-40ac-9561-fd0af0285e83 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.704875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d5e18-9d79-40ac-9561-fd0af0285e83 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.707031 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 386/782] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:47.708044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d5e18-9d79-40ac-9561-fd0af0285e83 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.709194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61db2d-347f-4e44-82d7-425ae2318cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:47.709826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61db2d-347f-4e44-82d7-425ae2318cae None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.710105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61db2d-347f-4e44-82d7-425ae2318cae None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.710378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61db2d-347f-4e44-82d7-425ae2318cae None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.711048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61db2d-347f-4e44-82d7-425ae2318cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:47.711386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61db2d-347f-4e44-82d7-425ae2318cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:47.713302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61db2d-347f-4e44-82d7-425ae2318cae None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:47.731351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 397/783] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:47.732955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818de8ea-2adb-4468-966a-58adb225769c None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.733344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818de8ea-2adb-4468-966a-58adb225769c None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.733629 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818de8ea-2adb-4468-966a-58adb225769c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.758524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-818de8ea-2adb-4468-966a-58adb225769c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:47.760341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-818de8ea-2adb-4468-966a-58adb225769c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:47.828268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:47.837017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:47.850190 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 398/784] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:47.851335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818de8ea-2adb-4468-966a-58adb225769c None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.855906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d5e18-9d79-40ac-9561-fd0af0285e83 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:47.938698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256b43b-c2e7-42e7-88ed-123098f6b6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:47.939685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256b43b-c2e7-42e7-88ed-123098f6b6ec None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.940117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256b43b-c2e7-42e7-88ed-123098f6b6ec None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.940538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256b43b-c2e7-42e7-88ed-123098f6b6ec None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.941326 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256b43b-c2e7-42e7-88ed-123098f6b6ec None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:47.941781 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256b43b-c2e7-42e7-88ed-123098f6b6ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:47.943373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256b43b-c2e7-42e7-88ed-123098f6b6ec None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:47.955390 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 387/785] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Aug 31 09:35:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:47.956619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256b43b-c2e7-42e7-88ed-123098f6b6ec None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.960865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818de8ea-2adb-4468-966a-58adb225769c None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:48.001956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603d179-4199-49e8-9d21-a3264d50a036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:48.002595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603d179-4199-49e8-9d21-a3264d50a036 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.002870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603d179-4199-49e8-9d21-a3264d50a036 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.003147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603d179-4199-49e8-9d21-a3264d50a036 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.011418 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603d179-4199-49e8-9d21-a3264d50a036 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:48.012874 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603d179-4199-49e8-9d21-a3264d50a036 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-9603d179-4199-49e8-9d21-a3264d50a036 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:48.036266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:48.037264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:48.038226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:48.038226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:48.038226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:48.038226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:48.038226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:48.038226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:48.038226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:48.038226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:48.038226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:48.038226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:48.038226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 399/786] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 09:35:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:48.044225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603d179-4199-49e8-9d21-a3264d50a036 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:48.155091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:48.169276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256b43b-c2e7-42e7-88ed-123098f6b6ec None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:48.219038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd234175-e017-4d5a-8237-92d577479fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:48.220405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd234175-e017-4d5a-8237-92d577479fa5 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.220940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd234175-e017-4d5a-8237-92d577479fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.221406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd234175-e017-4d5a-8237-92d577479fa5 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.222514 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd234175-e017-4d5a-8237-92d577479fa5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:48.223285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd234175-e017-4d5a-8237-92d577479fa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:48.225835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd234175-e017-4d5a-8237-92d577479fa5 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:48.233434 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 388/787] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Aug 31 09:35:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:48.239661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd234175-e017-4d5a-8237-92d577479fa5 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:48.241262 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603d179-4199-49e8-9d21-a3264d50a036 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:48.268530 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:48.289900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246e2b3-098f-4d3b-89e4-4901b77f7851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:48.291185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246e2b3-098f-4d3b-89e4-4901b77f7851 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.291582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246e2b3-098f-4d3b-89e4-4901b77f7851 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.291860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246e2b3-098f-4d3b-89e4-4901b77f7851 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.292706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246e2b3-098f-4d3b-89e4-4901b77f7851 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:48.293014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246e2b3-098f-4d3b-89e4-4901b77f7851 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:48.294548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246e2b3-098f-4d3b-89e4-4901b77f7851 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:48.319160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 400/788] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:48.320204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246e2b3-098f-4d3b-89e4-4901b77f7851 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:48.379167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd234175-e017-4d5a-8237-92d577479fa5 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:48.432878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a752b99-74cd-4623-9861-73b7b3539ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:48.434076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a752b99-74cd-4623-9861-73b7b3539ada None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.434640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a752b99-74cd-4623-9861-73b7b3539ada None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.435222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a752b99-74cd-4623-9861-73b7b3539ada None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.436750 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246e2b3-098f-4d3b-89e4-4901b77f7851 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:48.445583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:48.455662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a752b99-74cd-4623-9861-73b7b3539ada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-7a752b99-74cd-4623-9861-73b7b3539ada None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:48.467156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:48.468418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:48.469494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:48.469494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:48.469494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:48.469494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:48.469494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:48.469494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:48.469494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:48.469494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:48.469494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:48.469494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:48.469494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:48.469494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:48.471885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 389/789] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:48.473567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a752b99-74cd-4623-9861-73b7b3539ada None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:48.483751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb513e-c65d-4110-8546-e0db31dcefbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:48.484632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb513e-c65d-4110-8546-e0db31dcefbc None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.485054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb513e-c65d-4110-8546-e0db31dcefbc None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.485466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb513e-c65d-4110-8546-e0db31dcefbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.486304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb513e-c65d-4110-8546-e0db31dcefbc None admin] RBAC: Adding query filter params (interface=public, service_id=28652dc4cacb4d4898032a61ca4059d8, region_id=RegionOne) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:48.486762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb513e-c65d-4110-8546-e0db31dcefbc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:48.488324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb513e-c65d-4110-8546-e0db31dcefbc None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:48.498044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 401/790] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/endpoints?service_id=28652dc4cacb4d4898032a61ca4059d8&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:48.499404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb513e-c65d-4110-8546-e0db31dcefbc None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:48.610052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:48.612833 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb513e-c65d-4110-8546-e0db31dcefbc None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:48.636219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a752b99-74cd-4623-9861-73b7b3539ada None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:48.653370 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec760f-3da9-4a5c-afe2-61a71250f486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:48.654300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec760f-3da9-4a5c-afe2-61a71250f486 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.654813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec760f-3da9-4a5c-afe2-61a71250f486 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.655282 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec760f-3da9-4a5c-afe2-61a71250f486 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.656269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec760f-3da9-4a5c-afe2-61a71250f486 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:48.656712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec760f-3da9-4a5c-afe2-61a71250f486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:48.658636 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec760f-3da9-4a5c-afe2-61a71250f486 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:48.671403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 402/791] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:48.672669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec760f-3da9-4a5c-afe2-61a71250f486 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:48.677458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7fde5-34fb-43cb-b8ba-f84a98b1b84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:48.677746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7fde5-34fb-43cb-b8ba-f84a98b1b84c None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.678117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7fde5-34fb-43cb-b8ba-f84a98b1b84c None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.678459 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7fde5-34fb-43cb-b8ba-f84a98b1b84c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.698176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7fde5-34fb-43cb-b8ba-f84a98b1b84c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:48.699926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7fde5-34fb-43cb-b8ba-f84a98b1b84c None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:48.710367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-20c7fde5-34fb-43cb-b8ba-f84a98b1b84c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:48.716731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:48.718009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:48.719254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:48.719254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:48.719254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:48.723340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 390/792] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:48.724230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7fde5-34fb-43cb-b8ba-f84a98b1b84c None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:48.799397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec760f-3da9-4a5c-afe2-61a71250f486 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:48.837370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7fde5-34fb-43cb-b8ba-f84a98b1b84c None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:48.840413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d085a6-aef0-4a88-8ddc-a6d02cb54658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:48.841513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d085a6-aef0-4a88-8ddc-a6d02cb54658 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.842021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d085a6-aef0-4a88-8ddc-a6d02cb54658 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.842470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d085a6-aef0-4a88-8ddc-a6d02cb54658 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.843517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d085a6-aef0-4a88-8ddc-a6d02cb54658 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:48.845082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d085a6-aef0-4a88-8ddc-a6d02cb54658 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:48.851902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 403/793] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:48.855010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d085a6-aef0-4a88-8ddc-a6d02cb54658 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:48.881260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:48.904085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031b404-f370-45ad-8db2-da7f222da24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:48.904667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031b404-f370-45ad-8db2-da7f222da24d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.904951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031b404-f370-45ad-8db2-da7f222da24d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.905228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031b404-f370-45ad-8db2-da7f222da24d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.925639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031b404-f370-45ad-8db2-da7f222da24d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:48.927256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031b404-f370-45ad-8db2-da7f222da24d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-5031b404-f370-45ad-8db2-da7f222da24d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:48.954986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:48.956610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:48.957756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:48.957756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:48.957756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:48.957756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 391/794] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:48.961935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031b404-f370-45ad-8db2-da7f222da24d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:49.003094 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:49.012707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d085a6-aef0-4a88-8ddc-a6d02cb54658 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:49.063515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91929821-ab44-4721-8bfc-13d64c861382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:49.064430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91929821-ab44-4721-8bfc-13d64c861382 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.064845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91929821-ab44-4721-8bfc-13d64c861382 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.065244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91929821-ab44-4721-8bfc-13d64c861382 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.066128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91929821-ab44-4721-8bfc-13d64c861382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:49.066561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91929821-ab44-4721-8bfc-13d64c861382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:49.068260 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91929821-ab44-4721-8bfc-13d64c861382 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:49.077235 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031b404-f370-45ad-8db2-da7f222da24d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:49.099269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 404/795] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:49.104283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91929821-ab44-4721-8bfc-13d64c861382 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:49.123428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642afcf3-e8f1-4e86-b595-12c44d265df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:49.124139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642afcf3-e8f1-4e86-b595-12c44d265df6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.124418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642afcf3-e8f1-4e86-b595-12c44d265df6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.124754 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642afcf3-e8f1-4e86-b595-12c44d265df6 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.125414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642afcf3-e8f1-4e86-b595-12c44d265df6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:49.125769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642afcf3-e8f1-4e86-b595-12c44d265df6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:49.127211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642afcf3-e8f1-4e86-b595-12c44d265df6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:49.141583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 392/796] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:49.153048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642afcf3-e8f1-4e86-b595-12c44d265df6 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:49.200931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:49.273813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91929821-ab44-4721-8bfc-13d64c861382 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:49.282410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642afcf3-e8f1-4e86-b595-12c44d265df6 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:49.328189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac15ed-29c7-4bb4-ae9c-4e3b970f8a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:49.328934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac15ed-29c7-4bb4-ae9c-4e3b970f8a21 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.329214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac15ed-29c7-4bb4-ae9c-4e3b970f8a21 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.329487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac15ed-29c7-4bb4-ae9c-4e3b970f8a21 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.332045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14923852-1a07-4223-8b83-0dcf58b4917e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:49.332712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14923852-1a07-4223-8b83-0dcf58b4917e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.332987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14923852-1a07-4223-8b83-0dcf58b4917e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.333338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14923852-1a07-4223-8b83-0dcf58b4917e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.349395 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac15ed-29c7-4bb4-ae9c-4e3b970f8a21 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:49.350935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14923852-1a07-4223-8b83-0dcf58b4917e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:49.352560 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14923852-1a07-4223-8b83-0dcf58b4917e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:49.353764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac15ed-29c7-4bb4-ae9c-4e3b970f8a21 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-14923852-1a07-4223-8b83-0dcf58b4917e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:49.375872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:49.378069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:49.380266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:49.380266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:49.380266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:49.380266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:49.380266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:49.380266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:49.380266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:49.380266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-f1ac15ed-29c7-4bb4-ae9c-4e3b970f8a21 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:49.381283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:49.387622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:49.390322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:49.390322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:49.390322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:49.390322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:49.390322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:49.390322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:49.390322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:49.390322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:49.390322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:49.390322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:49.391670 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 405/797] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:49.391819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 393/798] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:49.391961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14923852-1a07-4223-8b83-0dcf58b4917e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:49.394452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac15ed-29c7-4bb4-ae9c-4e3b970f8a21 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:49.439587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:49.582546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14923852-1a07-4223-8b83-0dcf58b4917e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:49.600277 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:49.605849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac15ed-29c7-4bb4-ae9c-4e3b970f8a21 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:49.634650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44517cce-14ae-43ad-b900-17c419f4da13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:49.635599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44517cce-14ae-43ad-b900-17c419f4da13 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.636052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44517cce-14ae-43ad-b900-17c419f4da13 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.636476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44517cce-14ae-43ad-b900-17c419f4da13 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.637337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44517cce-14ae-43ad-b900-17c419f4da13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:49.637795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44517cce-14ae-43ad-b900-17c419f4da13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:49.639462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44517cce-14ae-43ad-b900-17c419f4da13 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:49.650491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1517caa-41fe-4952-84fd-c236c2de22aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:49.651712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1517caa-41fe-4952-84fd-c236c2de22aa None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.652159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1517caa-41fe-4952-84fd-c236c2de22aa None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.652686 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1517caa-41fe-4952-84fd-c236c2de22aa None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.653974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1517caa-41fe-4952-84fd-c236c2de22aa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:49.654496 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1517caa-41fe-4952-84fd-c236c2de22aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:49.660296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1517caa-41fe-4952-84fd-c236c2de22aa None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:49.660525 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 406/799] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:49.662110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c96027-a1fa-40a2-bcd8-9552a70e49a0 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.662552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c96027-a1fa-40a2-bcd8-9552a70e49a0 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.662999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c96027-a1fa-40a2-bcd8-9552a70e49a0 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.664807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 407/800] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:49.665881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c96027-a1fa-40a2-bcd8-9552a70e49a0 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:49.685566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 394/801] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Aug 31 09:35:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:49.693301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335c86c-19bc-46a4-9e7d-8d7acbfd4a50 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.693812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335c86c-19bc-46a4-9e7d-8d7acbfd4a50 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.694318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335c86c-19bc-46a4-9e7d-8d7acbfd4a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.719819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-7335c86c-19bc-46a4-9e7d-8d7acbfd4a50 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:49.721246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7335c86c-19bc-46a4-9e7d-8d7acbfd4a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:49.757846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:49.787625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c96027-a1fa-40a2-bcd8-9552a70e49a0 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:49.833451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43b2ca-db74-4a4f-b196-3a9ed8e4c2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:49.834057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43b2ca-db74-4a4f-b196-3a9ed8e4c2d1 None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.834336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43b2ca-db74-4a4f-b196-3a9ed8e4c2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.834611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43b2ca-db74-4a4f-b196-3a9ed8e4c2d1 None admin] PATH_INFO: `/v3/projects/8ef8b586382941408cd88ee654c42d32/users/0a2e5c1014f1441aa47d5de46462b6d9/roles/dcba9d07b31a4387b3014365e6e2ea10` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.853936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43b2ca-db74-4a4f-b196-3a9ed8e4c2d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef8b586382941408cd88ee654c42d32, user_id=0a2e5c1014f1441aa47d5de46462b6d9, role_id=dcba9d07b31a4387b3014365e6e2ea10)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:49.855477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43b2ca-db74-4a4f-b196-3a9ed8e4c2d1 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:49.873403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 408/802] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:49 2022] PUT /identity/v3/projects/8ef8b586382941408cd88ee654c42d32/users/0a2e5c1014f1441aa47d5de46462b6d9/roles/dcba9d07b31a4387b3014365e6e2ea10 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:35:49.880045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43b2ca-db74-4a4f-b196-3a9ed8e4c2d1 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:49.937733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 395/803] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:49.973505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ed44c-c416-4536-b69f-425a3f82578f None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.973946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ed44c-c416-4536-b69f-425a3f82578f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.974317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ed44c-c416-4536-b69f-425a3f82578f None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.976178 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 396/804] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:49.982397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a3c46-5c1d-4cea-a2c3-7ae333a2adfe None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.982711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a3c46-5c1d-4cea-a2c3-7ae333a2adfe None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.982951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a3c46-5c1d-4cea-a2c3-7ae333a2adfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:50.005131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-1a7a3c46-5c1d-4cea-a2c3-7ae333a2adfe None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:50.007039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1a7a3c46-5c1d-4cea-a2c3-7ae333a2adfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:50.116786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43b2ca-db74-4a4f-b196-3a9ed8e4c2d1 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:50.153151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:50.169094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 397/805] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:50.183019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c60767-ffc8-4bf2-b769-136f87a0d1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:50.183979 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c60767-ffc8-4bf2-b769-136f87a0d1e3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:50.184261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c60767-ffc8-4bf2-b769-136f87a0d1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:50.184564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c60767-ffc8-4bf2-b769-136f87a0d1e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:50.201530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c60767-ffc8-4bf2-b769-136f87a0d1e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:50.203014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c60767-ffc8-4bf2-b769-136f87a0d1e3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-e6c60767-ffc8-4bf2-b769-136f87a0d1e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:50.230280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:50.231336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:50.232212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:50.232212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:50.232212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:50.232396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 409/806] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:50.238659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a3c46-5c1d-4cea-a2c3-7ae333a2adfe None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:50.260526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:50.281675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c60767-ffc8-4bf2-b769-136f87a0d1e3 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:50.380354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a3c46-5c1d-4cea-a2c3-7ae333a2adfe None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:50.429738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c60767-ffc8-4bf2-b769-136f87a0d1e3 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:50.462879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5379965-1262-4078-8607-e0f1e938d872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:50.463523 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5379965-1262-4078-8607-e0f1e938d872 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:50.463875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5379965-1262-4078-8607-e0f1e938d872 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:50.464059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5379965-1262-4078-8607-e0f1e938d872 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:50.464707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5379965-1262-4078-8607-e0f1e938d872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:50.465002 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5379965-1262-4078-8607-e0f1e938d872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:50.466399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5379965-1262-4078-8607-e0f1e938d872 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:50.467200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb40358-f504-4f66-85f2-a657886e6f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:50.467844 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb40358-f504-4f66-85f2-a657886e6f59 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:50.468145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb40358-f504-4f66-85f2-a657886e6f59 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:50.468396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb40358-f504-4f66-85f2-a657886e6f59 None admin] PATH_INFO: `/v3/services/network` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:50.468979 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb40358-f504-4f66-85f2-a657886e6f59 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:50.470324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb40358-f504-4f66-85f2-a657886e6f59 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-2fb40358-f504-4f66-85f2-a657886e6f59 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:50.480013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:50.481016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:50.481925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 410/807] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:50 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:50.486919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb40358-f504-4f66-85f2-a657886e6f59 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:50.493652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 398/808] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:50.504164 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5379965-1262-4078-8607-e0f1e938d872 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:50.577785 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:50.603151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb40358-f504-4f66-85f2-a657886e6f59 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:50.643423 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e37f5-4ef4-4d94-ba39-9221db446589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:50.644051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e37f5-4ef4-4d94-ba39-9221db446589 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:50.644397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e37f5-4ef4-4d94-ba39-9221db446589 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:50.644508 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:50.644795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e37f5-4ef4-4d94-ba39-9221db446589 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:50.645503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e37f5-4ef4-4d94-ba39-9221db446589 None admin] RBAC: Adding query filter params (name=network) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:50.645878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e37f5-4ef4-4d94-ba39-9221db446589 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:50.647455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e37f5-4ef4-4d94-ba39-9221db446589 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:50.653357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 411/809] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:50.654327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e37f5-4ef4-4d94-ba39-9221db446589 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:50.671816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5379965-1262-4078-8607-e0f1e938d872 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:50.709895 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343cee1-ae4e-4f55-a4c7-93a717f8675b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:50.710647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343cee1-ae4e-4f55-a4c7-93a717f8675b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:50.710923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343cee1-ae4e-4f55-a4c7-93a717f8675b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:50.711210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343cee1-ae4e-4f55-a4c7-93a717f8675b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:50.720298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343cee1-ae4e-4f55-a4c7-93a717f8675b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:50.721915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343cee1-ae4e-4f55-a4c7-93a717f8675b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-e343cee1-ae4e-4f55-a4c7-93a717f8675b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:50.735560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:50.736706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:50.737679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:50.737679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:50.737679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:50.737679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:50.737679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:50.737679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:50.737679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:50.737679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:50.737679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 399/810] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:50.738176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343cee1-ae4e-4f55-a4c7-93a717f8675b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:50.772618 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e37f5-4ef4-4d94-ba39-9221db446589 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:50.781368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:50.812864 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7cbcf-e1bf-4a60-8464-b6c5e5aab18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:50.813524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7cbcf-e1bf-4a60-8464-b6c5e5aab18d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:50.813803 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7cbcf-e1bf-4a60-8464-b6c5e5aab18d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:50.814074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7cbcf-e1bf-4a60-8464-b6c5e5aab18d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:50.825835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7cbcf-e1bf-4a60-8464-b6c5e5aab18d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-40b7cbcf-e1bf-4a60-8464-b6c5e5aab18d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:50.831515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:50.833489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 412/811] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:50.835302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7cbcf-e1bf-4a60-8464-b6c5e5aab18d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:50.863602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343cee1-ae4e-4f55-a4c7-93a717f8675b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:50.904519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b603fa-caaf-4d06-a3a7-be469fb81039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:50.905177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b603fa-caaf-4d06-a3a7-be469fb81039 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:50.905451 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b603fa-caaf-4d06-a3a7-be469fb81039 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:50.905720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b603fa-caaf-4d06-a3a7-be469fb81039 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:50.906349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b603fa-caaf-4d06-a3a7-be469fb81039 None admin] RBAC: Adding query filter params (type=network) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:50.906685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b603fa-caaf-4d06-a3a7-be469fb81039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:50.909246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b603fa-caaf-4d06-a3a7-be469fb81039 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:50.916003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 400/812] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:50.917106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b603fa-caaf-4d06-a3a7-be469fb81039 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:50.920018 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:50.946060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7cbcf-e1bf-4a60-8464-b6c5e5aab18d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:50.982398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed9a2f-e6e6-4c13-ba83-5fe601d10e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:50.983014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed9a2f-e6e6-4c13-ba83-5fe601d10e70 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:50.983335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed9a2f-e6e6-4c13-ba83-5fe601d10e70 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:50.983608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed9a2f-e6e6-4c13-ba83-5fe601d10e70 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:50.984371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed9a2f-e6e6-4c13-ba83-5fe601d10e70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:50.984675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed9a2f-e6e6-4c13-ba83-5fe601d10e70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:50.986098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed9a2f-e6e6-4c13-ba83-5fe601d10e70 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:51.004305 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 413/813] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:51.005215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed9a2f-e6e6-4c13-ba83-5fe601d10e70 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:51.034272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:51.036963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b603fa-caaf-4d06-a3a7-be469fb81039 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:51.078368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78278a06-b703-4135-a2b9-94b0c7d1e6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:51.079165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78278a06-b703-4135-a2b9-94b0c7d1e6b1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:51.079587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78278a06-b703-4135-a2b9-94b0c7d1e6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:51.079999 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78278a06-b703-4135-a2b9-94b0c7d1e6b1 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:51.080999 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78278a06-b703-4135-a2b9-94b0c7d1e6b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:51.081537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78278a06-b703-4135-a2b9-94b0c7d1e6b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:51.083166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78278a06-b703-4135-a2b9-94b0c7d1e6b1 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:51.094296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 401/814] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:51.095573 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78278a06-b703-4135-a2b9-94b0c7d1e6b1 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:51.112398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed9a2f-e6e6-4c13-ba83-5fe601d10e70 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:51.149199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1971af7-c3fe-4291-bfe1-ea34b9fc5cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:51.149554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1971af7-c3fe-4291-bfe1-ea34b9fc5cbf None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:51.150338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1971af7-c3fe-4291-bfe1-ea34b9fc5cbf None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:51.150616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1971af7-c3fe-4291-bfe1-ea34b9fc5cbf None admin] PATH_INFO: `/v3/endpoints` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:51.151337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1971af7-c3fe-4291-bfe1-ea34b9fc5cbf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:51.152722 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1971af7-c3fe-4291-bfe1-ea34b9fc5cbf None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:51.171044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:51.179895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 414/815] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:35:51 2022] POST /identity/v3/endpoints => generated 337 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:51.181520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1971af7-c3fe-4291-bfe1-ea34b9fc5cbf None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:51.233745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78278a06-b703-4135-a2b9-94b0c7d1e6b1 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:51.331912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7891f1-d6c7-4105-96c3-b75980c3fa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:51.333908 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7891f1-d6c7-4105-96c3-b75980c3fa56 None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:51.334836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7891f1-d6c7-4105-96c3-b75980c3fa56 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:51.335387 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7891f1-d6c7-4105-96c3-b75980c3fa56 None admin] PATH_INFO: `/v3/projects/8ef8b586382941408cd88ee654c42d32/users/c5cf86ae159749cea0cc9783bffbfe75/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:51.338899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7891f1-d6c7-4105-96c3-b75980c3fa56 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef8b586382941408cd88ee654c42d32, user_id=c5cf86ae159749cea0cc9783bffbfe75, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:51.339995 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:51.342661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7891f1-d6c7-4105-96c3-b75980c3fa56 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:51.363483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 402/816] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:51 2022] PUT /identity/v3/projects/8ef8b586382941408cd88ee654c42d32/users/c5cf86ae159749cea0cc9783bffbfe75/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:35:51.364659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a977a2-25b5-4137-b956-f2cbe50de74a None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:51.364943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a977a2-25b5-4137-b956-f2cbe50de74a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:51.365215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a977a2-25b5-4137-b956-f2cbe50de74a None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:51.366602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 403/817] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:51.371362 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1971af7-c3fe-4291-bfe1-ea34b9fc5cbf None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:51.374226 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02132b-cb1e-4214-8759-ee036cadc91e None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:51.374733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02132b-cb1e-4214-8759-ee036cadc91e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:51.375011 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02132b-cb1e-4214-8759-ee036cadc91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:51.401513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-0c02132b-cb1e-4214-8759-ee036cadc91e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:51.402807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0c02132b-cb1e-4214-8759-ee036cadc91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:51.418867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e05cb3-1251-46af-9788-3e99b15d7aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:51.419960 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e05cb3-1251-46af-9788-3e99b15d7aed None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:51.420522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e05cb3-1251-46af-9788-3e99b15d7aed None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:51.421009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e05cb3-1251-46af-9788-3e99b15d7aed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:51.434398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e05cb3-1251-46af-9788-3e99b15d7aed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:51.436351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e05cb3-1251-46af-9788-3e99b15d7aed None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-80e05cb3-1251-46af-9788-3e99b15d7aed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:51.455019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:51.456127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:51.457027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:51.457027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:51.457027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:51.459362 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 415/818] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:51.467245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e05cb3-1251-46af-9788-3e99b15d7aed None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:51.498854 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 404/819] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:51.580350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e05cb3-1251-46af-9788-3e99b15d7aed None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:51.616654 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52624ee-dfbc-44be-a927-25c35ec01d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:51.617276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52624ee-dfbc-44be-a927-25c35ec01d32 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:51.617542 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52624ee-dfbc-44be-a927-25c35ec01d32 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:51.617815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52624ee-dfbc-44be-a927-25c35ec01d32 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:51.618541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52624ee-dfbc-44be-a927-25c35ec01d32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:51.619535 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52624ee-dfbc-44be-a927-25c35ec01d32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:51.620932 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52624ee-dfbc-44be-a927-25c35ec01d32 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:51.622448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:51.637350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 416/820] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:51.645585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02132b-cb1e-4214-8759-ee036cadc91e None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:51.697491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:51.749910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02132b-cb1e-4214-8759-ee036cadc91e None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:51.787146 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451b847-7819-46e4-81ee-98b2b38dd9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:51.787792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451b847-7819-46e4-81ee-98b2b38dd9a3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:51.788072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451b847-7819-46e4-81ee-98b2b38dd9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:51.788338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451b847-7819-46e4-81ee-98b2b38dd9a3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:51.794858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52624ee-dfbc-44be-a927-25c35ec01d32 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:51.796878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451b847-7819-46e4-81ee-98b2b38dd9a3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:51.798341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451b847-7819-46e4-81ee-98b2b38dd9a3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-e451b847-7819-46e4-81ee-98b2b38dd9a3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:51.811641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:51.812685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:51.814351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:51.814351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:51.814351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:51.814351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:51.814351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:51.814351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:51.814351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:51.814351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:51.814351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:51.814351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:51.814351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 405/821] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 09:35:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:51.818667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451b847-7819-46e4-81ee-98b2b38dd9a3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:51.915070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52624ee-dfbc-44be-a927-25c35ec01d32 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:51.932723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451b847-7819-46e4-81ee-98b2b38dd9a3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:51.951297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:51.959856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a6b88-44b5-4aae-b1e8-229cb0338cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:51.960522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a6b88-44b5-4aae-b1e8-229cb0338cbf None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:51.960952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a6b88-44b5-4aae-b1e8-229cb0338cbf None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:51.961364 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a6b88-44b5-4aae-b1e8-229cb0338cbf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:51.961969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a6b88-44b5-4aae-b1e8-229cb0338cbf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:51.963952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a6b88-44b5-4aae-b1e8-229cb0338cbf None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:51.974742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57af2d9-ebc4-4d41-9c71-88bcf532ca24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:51.975536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57af2d9-ebc4-4d41-9c71-88bcf532ca24 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-0f5a6b88-44b5-4aae-b1e8-229cb0338cbf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:51.975729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:51.976751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:35:51.977815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:51.978753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57af2d9-ebc4-4d41-9c71-88bcf532ca24 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:51.978753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57af2d9-ebc4-4d41-9c71-88bcf532ca24 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:51.978902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 417/822] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Aug 31 09:35:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:51.978979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57af2d9-ebc4-4d41-9c71-88bcf532ca24 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:51.978979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57af2d9-ebc4-4d41-9c71-88bcf532ca24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:51.979141 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57af2d9-ebc4-4d41-9c71-88bcf532ca24 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:51.985518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a6b88-44b5-4aae-b1e8-229cb0338cbf None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:52.003205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 406/823] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 09:35:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:52.004332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57af2d9-ebc4-4d41-9c71-88bcf532ca24 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:52.056430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:52.125152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a6b88-44b5-4aae-b1e8-229cb0338cbf None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:52.132781 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57af2d9-ebc4-4d41-9c71-88bcf532ca24 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:52.171261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c97d8-853b-409c-b245-ca01533958da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:52.171961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c97d8-853b-409c-b245-ca01533958da None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:52.172283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c97d8-853b-409c-b245-ca01533958da None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:52.172512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c97d8-853b-409c-b245-ca01533958da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:52.175105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430a518-8dc7-45d7-b3f6-e14216dfe58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:52.177437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430a518-8dc7-45d7-b3f6-e14216dfe58d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:52.177957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430a518-8dc7-45d7-b3f6-e14216dfe58d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:52.178457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430a518-8dc7-45d7-b3f6-e14216dfe58d None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:52.179333 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430a518-8dc7-45d7-b3f6-e14216dfe58d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:52.179904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430a518-8dc7-45d7-b3f6-e14216dfe58d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:52.181598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430a518-8dc7-45d7-b3f6-e14216dfe58d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:52.193045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 418/824] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Aug 31 09:35:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:52.195260 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430a518-8dc7-45d7-b3f6-e14216dfe58d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:52.215776 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:52.253855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c97d8-853b-409c-b245-ca01533958da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:52.255172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c97d8-853b-409c-b245-ca01533958da None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:52.312265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430a518-8dc7-45d7-b3f6-e14216dfe58d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:52.339586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:52.356966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca6637-24a6-4ae2-ac73-85a752d6cce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:52.357752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca6637-24a6-4ae2-ac73-85a752d6cce0 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:52.358237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca6637-24a6-4ae2-ac73-85a752d6cce0 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:52.358693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca6637-24a6-4ae2-ac73-85a752d6cce0 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:52.359605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca6637-24a6-4ae2-ac73-85a752d6cce0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:52.360084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca6637-24a6-4ae2-ac73-85a752d6cce0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:52.361681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca6637-24a6-4ae2-ac73-85a752d6cce0 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:52.363189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 407/825] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:35:52.368799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade01f3e-d457-479d-84cb-547f5fa93f18 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:52.369410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade01f3e-d457-479d-84cb-547f5fa93f18 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:52.369539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 419/826] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Aug 31 09:35:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:52.369622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade01f3e-d457-479d-84cb-547f5fa93f18 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:52.371155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 408/827] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:52.374920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca6637-24a6-4ae2-ac73-85a752d6cce0 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:52.376224 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60fa94-0ea8-4411-b7ff-2134c8beef9b None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:52.376504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60fa94-0ea8-4411-b7ff-2134c8beef9b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:52.376795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60fa94-0ea8-4411-b7ff-2134c8beef9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:52.396558 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ac60fa94-0ea8-4411-b7ff-2134c8beef9b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:52.397739 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ac60fa94-0ea8-4411-b7ff-2134c8beef9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:52.466609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:52.472588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 409/828] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:52.473403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60fa94-0ea8-4411-b7ff-2134c8beef9b None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:52.498772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca6637-24a6-4ae2-ac73-85a752d6cce0 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:52.541807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db4396-4939-4005-bf2f-e012cce05ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:52.542369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db4396-4939-4005-bf2f-e012cce05ef5 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:52.542651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db4396-4939-4005-bf2f-e012cce05ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:52.542927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db4396-4939-4005-bf2f-e012cce05ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:52.557899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db4396-4939-4005-bf2f-e012cce05ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:52.559503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db4396-4939-4005-bf2f-e012cce05ef5 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-90db4396-4939-4005-bf2f-e012cce05ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:52.580043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:52.581080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:52.581951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:52.581951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:52.581951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:52.581951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 420/829] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:52.586548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db4396-4939-4005-bf2f-e012cce05ef5 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:52.595550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60fa94-0ea8-4411-b7ff-2134c8beef9b None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:52.642726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25137600-c6c3-47ac-8a0b-5e2ec95f8d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:52.643366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25137600-c6c3-47ac-8a0b-5e2ec95f8d1c None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:52.643655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25137600-c6c3-47ac-8a0b-5e2ec95f8d1c None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:52.643962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25137600-c6c3-47ac-8a0b-5e2ec95f8d1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:52.644627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25137600-c6c3-47ac-8a0b-5e2ec95f8d1c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:52.645643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:52.647324 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25137600-c6c3-47ac-8a0b-5e2ec95f8d1c None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:52.679742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 410/830] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:35:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:52.730997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db4396-4939-4005-bf2f-e012cce05ef5 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:52.777545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25137600-c6c3-47ac-8a0b-5e2ec95f8d1c None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:52.782682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5aa78-2807-42ab-901e-31bde006cd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:52.783580 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5aa78-2807-42ab-901e-31bde006cd7f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:52.784005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5aa78-2807-42ab-901e-31bde006cd7f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:52.784540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5aa78-2807-42ab-901e-31bde006cd7f None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:52.785432 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e5aa78-2807-42ab-901e-31bde006cd7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:52.785953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e5aa78-2807-42ab-901e-31bde006cd7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:52.787627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e5aa78-2807-42ab-901e-31bde006cd7f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:52.804088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 421/831] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:52.811163 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5aa78-2807-42ab-901e-31bde006cd7f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:52.854766 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:52.938614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:52.940265 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25137600-c6c3-47ac-8a0b-5e2ec95f8d1c None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:52.950703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5aa78-2807-42ab-901e-31bde006cd7f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:52.983916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb518d-f45a-4c7b-9522-216c301c44cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:52.984458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb518d-f45a-4c7b-9522-216c301c44cb None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:52.984738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb518d-f45a-4c7b-9522-216c301c44cb None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:52.985010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb518d-f45a-4c7b-9522-216c301c44cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:52.997222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb518d-f45a-4c7b-9522-216c301c44cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:52.997390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f602f-006d-4254-98a7-1cd3274e141d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:52.997993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f602f-006d-4254-98a7-1cd3274e141d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:52.998281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f602f-006d-4254-98a7-1cd3274e141d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:52.998555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f602f-006d-4254-98a7-1cd3274e141d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-bcbb518d-f45a-4c7b-9522-216c301c44cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:53.002810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:53.003768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:53.004583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 411/832] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:53.011307 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f602f-006d-4254-98a7-1cd3274e141d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:53.011456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb518d-f45a-4c7b-9522-216c301c44cb None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:53.013115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f602f-006d-4254-98a7-1cd3274e141d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-945f602f-006d-4254-98a7-1cd3274e141d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:53.026739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:53.027902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:53.028963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:53.028963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:53.028963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:53.028963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:53.028963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:53.028963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:53.028963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:53.028963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:53.035145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 422/833] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:53.037577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f602f-006d-4254-98a7-1cd3274e141d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:53.126577 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb518d-f45a-4c7b-9522-216c301c44cb None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:53.141980 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:53.165145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f602f-006d-4254-98a7-1cd3274e141d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:53.166895 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30811305-495e-4699-8192-d2e6cd4ff93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:53.167663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30811305-495e-4699-8192-d2e6cd4ff93c None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:53.168044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30811305-495e-4699-8192-d2e6cd4ff93c None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:53.168436 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30811305-495e-4699-8192-d2e6cd4ff93c None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:53.169722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30811305-495e-4699-8192-d2e6cd4ff93c None admin] RBAC: Adding query filter params (name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:53.169722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30811305-495e-4699-8192-d2e6cd4ff93c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:53.171204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30811305-495e-4699-8192-d2e6cd4ff93c None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:53.185990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 412/834] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:53.187590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30811305-495e-4699-8192-d2e6cd4ff93c None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:53.211315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753daf1-1d2d-4bc2-9d0a-ffa4077a52a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:53.213231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753daf1-1d2d-4bc2-9d0a-ffa4077a52a9 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:53.213715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753daf1-1d2d-4bc2-9d0a-ffa4077a52a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:53.214169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753daf1-1d2d-4bc2-9d0a-ffa4077a52a9 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:53.215061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753daf1-1d2d-4bc2-9d0a-ffa4077a52a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:53.215608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753daf1-1d2d-4bc2-9d0a-ffa4077a52a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:53.217251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753daf1-1d2d-4bc2-9d0a-ffa4077a52a9 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:53.307658 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30811305-495e-4699-8192-d2e6cd4ff93c None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:53.347010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52ba9c-e981-4fc6-aaa6-20d406e9121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:53.347834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52ba9c-e981-4fc6-aaa6-20d406e9121a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:53.348192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52ba9c-e981-4fc6-aaa6-20d406e9121a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:53.348550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52ba9c-e981-4fc6-aaa6-20d406e9121a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:53.358349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 423/835] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:53.359682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753daf1-1d2d-4bc2-9d0a-ffa4077a52a9 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:53.362282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52ba9c-e981-4fc6-aaa6-20d406e9121a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:53.363795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52ba9c-e981-4fc6-aaa6-20d406e9121a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:53.378571 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-fa52ba9c-e981-4fc6-aaa6-20d406e9121a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:53.380254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:53.381672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:53.382699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:53.382699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:53.382699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:53.386941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 413/836] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:53.388504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52ba9c-e981-4fc6-aaa6-20d406e9121a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:53.419666 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:53.476800 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753daf1-1d2d-4bc2-9d0a-ffa4077a52a9 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:53.497947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52ba9c-e981-4fc6-aaa6-20d406e9121a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:53.528376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514db0b-0ef6-412d-9827-25a14655ec8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:53.529026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514db0b-0ef6-412d-9827-25a14655ec8f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:53.529390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514db0b-0ef6-412d-9827-25a14655ec8f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:53.529669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514db0b-0ef6-412d-9827-25a14655ec8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:53.530413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514db0b-0ef6-412d-9827-25a14655ec8f None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=ef72bb95992c4a21ae2dbb2a08c51421) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:53.530721 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514db0b-0ef6-412d-9827-25a14655ec8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:53.532245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514db0b-0ef6-412d-9827-25a14655ec8f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:53.538919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 424/837] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:53 2022] GET /identity/v3/role_assignments?user.id=ef72bb95992c4a21ae2dbb2a08c51421&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:53.540441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e0edd-2642-4072-8c96-4e335b64e542 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:53.541233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e0edd-2642-4072-8c96-4e335b64e542 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:53.541504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e0edd-2642-4072-8c96-4e335b64e542 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:53.547050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 425/838] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:53.549265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25b205-854e-4820-b828-7727fbc84b33 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:53.549676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25b205-854e-4820-b828-7727fbc84b33 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:53.550059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25b205-854e-4820-b828-7727fbc84b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:53.551055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f0087-c23d-42d1-b67b-4464e509c3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:53.552024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f0087-c23d-42d1-b67b-4464e509c3f3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:53.552478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f0087-c23d-42d1-b67b-4464e509c3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:53.552901 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f0087-c23d-42d1-b67b-4464e509c3f3 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:53.553840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f0087-c23d-42d1-b67b-4464e509c3f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:53.554303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f0087-c23d-42d1-b67b-4464e509c3f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:53.556215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f0087-c23d-42d1-b67b-4464e509c3f3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:53.575512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 414/839] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:53.576700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-8e25b205-854e-4820-b828-7727fbc84b33 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:53.578061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-8e25b205-854e-4820-b828-7727fbc84b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:53.580519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f0087-c23d-42d1-b67b-4464e509c3f3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:53.684969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 426/840] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:53.695603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:53.710211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f0087-c23d-42d1-b67b-4464e509c3f3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:53.755444 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a3e16-1e94-435a-b2a5-4716e2b38126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:53.756758 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a3e16-1e94-435a-b2a5-4716e2b38126 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:53.757450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a3e16-1e94-435a-b2a5-4716e2b38126 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:53.758109 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a3e16-1e94-435a-b2a5-4716e2b38126 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:53.772329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a3e16-1e94-435a-b2a5-4716e2b38126 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:53.775801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a3e16-1e94-435a-b2a5-4716e2b38126 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-e20a3e16-1e94-435a-b2a5-4716e2b38126 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:53.789848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:53.792078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:53.793140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:53.793140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:53.793140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:53.793140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:53.793140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:53.793140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:53.793140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:53.793140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:53.793140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:53.793140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:53.793140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 415/841] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 09:35:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:53.797621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25b205-854e-4820-b828-7727fbc84b33 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:53.810844 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:53.913732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25b205-854e-4820-b828-7727fbc84b33 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:53.952141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44888e9-18eb-4206-bc43-f2a87aea71e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:53.952754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44888e9-18eb-4206-bc43-f2a87aea71e4 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:53.953030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44888e9-18eb-4206-bc43-f2a87aea71e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:53.953297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44888e9-18eb-4206-bc43-f2a87aea71e4 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:53.954025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44888e9-18eb-4206-bc43-f2a87aea71e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:53.954323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44888e9-18eb-4206-bc43-f2a87aea71e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:53.955758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44888e9-18eb-4206-bc43-f2a87aea71e4 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:53.970312 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a3e16-1e94-435a-b2a5-4716e2b38126 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:53.975787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 427/842] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Aug 31 09:35:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:53.981569 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44888e9-18eb-4206-bc43-f2a87aea71e4 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:54.078992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:54.095410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a3e16-1e94-435a-b2a5-4716e2b38126 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:54.125802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44888e9-18eb-4206-bc43-f2a87aea71e4 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:54.149137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ce930-5721-4a8a-98da-0812e047e501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:54.150789 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ce930-5721-4a8a-98da-0812e047e501 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:54.151081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ce930-5721-4a8a-98da-0812e047e501 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:54.151387 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ce930-5721-4a8a-98da-0812e047e501 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:54.164278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479d099-df5a-41cb-bd64-c721fb604f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:54.164816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479d099-df5a-41cb-bd64-c721fb604f6f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:54.165097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479d099-df5a-41cb-bd64-c721fb604f6f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:54.165928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479d099-df5a-41cb-bd64-c721fb604f6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:54.166394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ce930-5721-4a8a-98da-0812e047e501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-eb7ce930-5721-4a8a-98da-0812e047e501 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:54.173649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:54.174612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:54.175512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:54.175512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:54.175512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:54.175512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:54.175512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:54.175512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:54.175512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:54.175512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:54.175512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:54.175512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:54.175512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:54.175512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:54.176143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 416/843] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:54.181376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ce930-5721-4a8a-98da-0812e047e501 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:54.195557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:54.257105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d479d099-df5a-41cb-bd64-c721fb604f6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:54.259865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d479d099-df5a-41cb-bd64-c721fb604f6f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:54.307579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ce930-5721-4a8a-98da-0812e047e501 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:54.337989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:54.355455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685b823-0d16-4f74-9416-38c0dfc2bd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:54.358309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685b823-0d16-4f74-9416-38c0dfc2bd40 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:54.359036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685b823-0d16-4f74-9416-38c0dfc2bd40 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:54.359474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685b823-0d16-4f74-9416-38c0dfc2bd40 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:54.360490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685b823-0d16-4f74-9416-38c0dfc2bd40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:54.360941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685b823-0d16-4f74-9416-38c0dfc2bd40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:54.362770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685b823-0d16-4f74-9416-38c0dfc2bd40 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:54.375493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 417/844] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:54.380652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685b823-0d16-4f74-9416-38c0dfc2bd40 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:54.384646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 428/845] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:53 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:35:54.391303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479d099-df5a-41cb-bd64-c721fb604f6f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:54.463426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:54.506236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685b823-0d16-4f74-9416-38c0dfc2bd40 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:54.536985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479d099-df5a-41cb-bd64-c721fb604f6f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:54.543786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c80072-b698-48da-bc27-6648bd9454fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:54.544421 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c80072-b698-48da-bc27-6648bd9454fb None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:54.544894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c80072-b698-48da-bc27-6648bd9454fb None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:54.545172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c80072-b698-48da-bc27-6648bd9454fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:54.557538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c80072-b698-48da-bc27-6648bd9454fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:54.559212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c80072-b698-48da-bc27-6648bd9454fb None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-97c80072-b698-48da-bc27-6648bd9454fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:54.574877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:54.576061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:54.576943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:54.576943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:54.576943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:54.581048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 418/846] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:54.581931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c80072-b698-48da-bc27-6648bd9454fb None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:54.588164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686efe7b-4809-4d0a-804d-6e7fe977c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:54.589147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686efe7b-4809-4d0a-804d-6e7fe977c0a4 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:54.589658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686efe7b-4809-4d0a-804d-6e7fe977c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:54.590135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686efe7b-4809-4d0a-804d-6e7fe977c0a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:54.606266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686efe7b-4809-4d0a-804d-6e7fe977c0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:54.609148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686efe7b-4809-4d0a-804d-6e7fe977c0a4 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-686efe7b-4809-4d0a-804d-6e7fe977c0a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:54.631557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:54.632548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:54.633421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:54.633421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:54.633421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:54.636963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 429/847] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:54.642486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686efe7b-4809-4d0a-804d-6e7fe977c0a4 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:54.669832 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:54.715575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c80072-b698-48da-bc27-6648bd9454fb None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:54.746504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:54.759817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43192373-277f-46cb-a66c-93e6d6d031d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:54.760500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43192373-277f-46cb-a66c-93e6d6d031d9 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:54.760778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43192373-277f-46cb-a66c-93e6d6d031d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:54.761043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43192373-277f-46cb-a66c-93e6d6d031d9 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:54.761687 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43192373-277f-46cb-a66c-93e6d6d031d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:54.761988 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43192373-277f-46cb-a66c-93e6d6d031d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:54.763386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43192373-277f-46cb-a66c-93e6d6d031d9 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:54.770340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686efe7b-4809-4d0a-804d-6e7fe977c0a4 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:54.781936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 419/848] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:54.781936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1a42b-6907-4c32-acea-309fd48bd5ea None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:54.781936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1a42b-6907-4c32-acea-309fd48bd5ea None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:54.781936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1a42b-6907-4c32-acea-309fd48bd5ea None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:54.781936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 420/849] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:54.783494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1a42b-6907-4c32-acea-309fd48bd5ea None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:54.814465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c68042-d585-45d4-8f3b-2dba1ff41b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:54.815092 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c68042-d585-45d4-8f3b-2dba1ff41b8d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:54.815404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c68042-d585-45d4-8f3b-2dba1ff41b8d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:54.815677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c68042-d585-45d4-8f3b-2dba1ff41b8d None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:54.816344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c68042-d585-45d4-8f3b-2dba1ff41b8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:54.816725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c68042-d585-45d4-8f3b-2dba1ff41b8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:54.818204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c68042-d585-45d4-8f3b-2dba1ff41b8d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:54.837162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 430/850] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:54.837162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18487a-8b9d-452f-a864-e35a59829c84 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:54.837162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18487a-8b9d-452f-a864-e35a59829c84 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:54.837162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18487a-8b9d-452f-a864-e35a59829c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:54.853918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-bf18487a-8b9d-452f-a864-e35a59829c84 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:54.855111 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-bf18487a-8b9d-452f-a864-e35a59829c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:54.938445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 431/851] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:54.939695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18487a-8b9d-452f-a864-e35a59829c84 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:54.950188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:54.953780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1a42b-6907-4c32-acea-309fd48bd5ea None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:55.012751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7f356-a29c-4c6a-93e6-3e6bb90912bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:55.014417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7f356-a29c-4c6a-93e6-3e6bb90912bb None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.015421 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7f356-a29c-4c6a-93e6-3e6bb90912bb None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.016383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7f356-a29c-4c6a-93e6-3e6bb90912bb None admin] PATH_INFO: `/v3/users/swift` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.032947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7f356-a29c-4c6a-93e6-3e6bb90912bb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:55.035794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7f356-a29c-4c6a-93e6-3e6bb90912bb None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-01c7f356-a29c-4c6a-93e6-3e6bb90912bb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:55.066640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:55.079105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:55.080107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:55.080107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:55.080107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:55.080107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:55.080107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:55.080107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:55.080107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:55.080107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:55.080107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:55.080107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:55.080107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 421/852] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:55.083696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7f356-a29c-4c6a-93e6-3e6bb90912bb None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:55.092499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18487a-8b9d-452f-a864-e35a59829c84 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:55.129388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8f099-d9eb-4073-8a5c-ddf6161d2a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:55.130039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8f099-d9eb-4073-8a5c-ddf6161d2a1a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.130321 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8f099-d9eb-4073-8a5c-ddf6161d2a1a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.130595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8f099-d9eb-4073-8a5c-ddf6161d2a1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.138551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8f099-d9eb-4073-8a5c-ddf6161d2a1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:55.140637 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8f099-d9eb-4073-8a5c-ddf6161d2a1a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:55.144875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-75a8f099-d9eb-4073-8a5c-ddf6161d2a1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:55.154252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:55.155224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:55.156120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:55.156120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:55.156120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:55.156120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:55.156120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:55.156120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:55.156120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:55.156120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:55.160462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 432/853] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:55.161378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8f099-d9eb-4073-8a5c-ddf6161d2a1a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:55.226545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7f356-a29c-4c6a-93e6-3e6bb90912bb None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:55.238283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:55.268460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b60300-dc97-49c0-bf91-bc86e6dd8850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:55.269156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b60300-dc97-49c0-bf91-bc86e6dd8850 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.269436 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b60300-dc97-49c0-bf91-bc86e6dd8850 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.269707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b60300-dc97-49c0-bf91-bc86e6dd8850 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.270439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b60300-dc97-49c0-bf91-bc86e6dd8850 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:55.270743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b60300-dc97-49c0-bf91-bc86e6dd8850 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:55.272207 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b60300-dc97-49c0-bf91-bc86e6dd8850 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:55.281139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8f099-d9eb-4073-8a5c-ddf6161d2a1a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:55.291261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 422/854] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Aug 31 09:35:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:55.292092 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b60300-dc97-49c0-bf91-bc86e6dd8850 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:55.329544 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b17b2-627f-47cc-949c-4da920f47b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:55.331279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b17b2-627f-47cc-949c-4da920f47b22 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.331731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b17b2-627f-47cc-949c-4da920f47b22 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.332212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b17b2-627f-47cc-949c-4da920f47b22 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.333059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b17b2-627f-47cc-949c-4da920f47b22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:55.333509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b17b2-627f-47cc-949c-4da920f47b22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:55.335088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b17b2-627f-47cc-949c-4da920f47b22 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:55.354890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 433/855] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:55.354890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b17b2-627f-47cc-949c-4da920f47b22 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:55.404565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:55.408786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b60300-dc97-49c0-bf91-bc86e6dd8850 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:55.447529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da4943-d078-4f92-a446-820572181f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:55.448024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da4943-d078-4f92-a446-820572181f7b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.448303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da4943-d078-4f92-a446-820572181f7b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.448684 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da4943-d078-4f92-a446-820572181f7b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.449271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da4943-d078-4f92-a446-820572181f7b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:55.450638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da4943-d078-4f92-a446-820572181f7b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-a7da4943-d078-4f92-a446-820572181f7b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:55.459800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:55.463254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:55.464161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 423/856] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:55.464964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da4943-d078-4f92-a446-820572181f7b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:55.469586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b17b2-627f-47cc-949c-4da920f47b22 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:55.500156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:55.511851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794ae94-18e3-405d-9fc2-fe7414746696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:55.512581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794ae94-18e3-405d-9fc2-fe7414746696 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.512860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794ae94-18e3-405d-9fc2-fe7414746696 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.513128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794ae94-18e3-405d-9fc2-fe7414746696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.574692 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da4943-d078-4f92-a446-820572181f7b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:55.589794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794ae94-18e3-405d-9fc2-fe7414746696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:55.591357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794ae94-18e3-405d-9fc2-fe7414746696 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:55.613473 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1f91e-68c7-4eb5-b736-c02324d68caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:55.614075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1f91e-68c7-4eb5-b736-c02324d68caa None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.614354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1f91e-68c7-4eb5-b736-c02324d68caa None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.614623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1f91e-68c7-4eb5-b736-c02324d68caa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.615376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1f91e-68c7-4eb5-b736-c02324d68caa None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=0a2e5c1014f1441aa47d5de46462b6d9) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:55.615677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1f91e-68c7-4eb5-b736-c02324d68caa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:55.617102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1f91e-68c7-4eb5-b736-c02324d68caa None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:55.624942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 424/857] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:55 2022] GET /identity/v3/role_assignments?user.id=0a2e5c1014f1441aa47d5de46462b6d9&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:55.625762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1f91e-68c7-4eb5-b736-c02324d68caa None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:55.667491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:55.728628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 434/858] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:55.730405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36bfa3-af3d-47a2-9861-d77b7c55be54 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.730763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36bfa3-af3d-47a2-9861-d77b7c55be54 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.731113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36bfa3-af3d-47a2-9861-d77b7c55be54 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.732788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 435/859] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:55.734356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36bfa3-af3d-47a2-9861-d77b7c55be54 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:55.764497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1f91e-68c7-4eb5-b736-c02324d68caa None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:55.768791 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:55.803144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0e95b-a9ad-4642-9414-ba344b96f2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:55.803784 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0e95b-a9ad-4642-9414-ba344b96f2ac None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.804064 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0e95b-a9ad-4642-9414-ba344b96f2ac None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.804337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0e95b-a9ad-4642-9414-ba344b96f2ac None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.804967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0e95b-a9ad-4642-9414-ba344b96f2ac None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:55.805269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0e95b-a9ad-4642-9414-ba344b96f2ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:55.806941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0e95b-a9ad-4642-9414-ba344b96f2ac None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:55.812310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 425/860] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:55.813508 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577fd9b-3163-49c2-be9a-d54b8ec54b96 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.813794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577fd9b-3163-49c2-be9a-d54b8ec54b96 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.814068 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577fd9b-3163-49c2-be9a-d54b8ec54b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.836899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b577fd9b-3163-49c2-be9a-d54b8ec54b96 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:55.838653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b577fd9b-3163-49c2-be9a-d54b8ec54b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:55.857141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36bfa3-af3d-47a2-9861-d77b7c55be54 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:55.899764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b51752-9328-4d32-8098-4638e06ee110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:55.900656 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b51752-9328-4d32-8098-4638e06ee110 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.900944 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b51752-9328-4d32-8098-4638e06ee110 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.901215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b51752-9328-4d32-8098-4638e06ee110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.912889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b51752-9328-4d32-8098-4638e06ee110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:55.914324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b51752-9328-4d32-8098-4638e06ee110 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:55.922054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 426/861] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:55.922054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577fd9b-3163-49c2-be9a-d54b8ec54b96 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-d8b51752-9328-4d32-8098-4638e06ee110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:55.930325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:55.932011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:55.933861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:55.933861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:55.933861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:55.933861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 436/862] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:55.937862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b51752-9328-4d32-8098-4638e06ee110 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:55.971573 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:56.032588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:56.063385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577fd9b-3163-49c2-be9a-d54b8ec54b96 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:56.065752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b51752-9328-4d32-8098-4638e06ee110 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:56.108237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca99c5-2aba-4116-9196-5a6ff572e89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:56.108932 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca99c5-2aba-4116-9196-5a6ff572e89a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:56.109494 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca99c5-2aba-4116-9196-5a6ff572e89a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:56.109777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca99c5-2aba-4116-9196-5a6ff572e89a None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:56.110521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca99c5-2aba-4116-9196-5a6ff572e89a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:56.110918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca99c5-2aba-4116-9196-5a6ff572e89a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:56.112534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca99c5-2aba-4116-9196-5a6ff572e89a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:56.119083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45b65d-02a6-477e-a24a-671fe7576fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:56.120159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45b65d-02a6-477e-a24a-671fe7576fe8 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:56.121313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45b65d-02a6-477e-a24a-671fe7576fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:56.121746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45b65d-02a6-477e-a24a-671fe7576fe8 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:56.122534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45b65d-02a6-477e-a24a-671fe7576fe8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:56.123456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45b65d-02a6-477e-a24a-671fe7576fe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:56.124968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45b65d-02a6-477e-a24a-671fe7576fe8 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:56.138514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 437/864] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:56.138514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca99c5-2aba-4116-9196-5a6ff572e89a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:56.139384 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 427/864] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:55 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:56.211367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45b65d-02a6-477e-a24a-671fe7576fe8 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:56.263819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca99c5-2aba-4116-9196-5a6ff572e89a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:56.304557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f6af2-0733-4dd7-8ffd-04a07b2a780a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:56.306308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f6af2-0733-4dd7-8ffd-04a07b2a780a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:56.306944 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f6af2-0733-4dd7-8ffd-04a07b2a780a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:56.307259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f6af2-0733-4dd7-8ffd-04a07b2a780a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:56.310834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:56.315990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f6af2-0733-4dd7-8ffd-04a07b2a780a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:56.318486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f6af2-0733-4dd7-8ffd-04a07b2a780a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:56.320693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-200f6af2-0733-4dd7-8ffd-04a07b2a780a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:56.331378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:56.341886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:56.343037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:56.343037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:56.343037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:56.343037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:56.343037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:56.343037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:56.343037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:56.343037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:56.343037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 438/865] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:56.343037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f6af2-0733-4dd7-8ffd-04a07b2a780a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:56.350930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45b65d-02a6-477e-a24a-671fe7576fe8 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:56.389380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a97176-9127-4de6-b091-eb460d1882a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:56.390519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a97176-9127-4de6-b091-eb460d1882a0 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:56.390947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a97176-9127-4de6-b091-eb460d1882a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:56.391407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a97176-9127-4de6-b091-eb460d1882a0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:56.392143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a97176-9127-4de6-b091-eb460d1882a0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:56.393780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a97176-9127-4de6-b091-eb460d1882a0 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-c5a97176-9127-4de6-b091-eb460d1882a0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:56.404193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:56.405875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:35:56.406964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:56.409534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 428/866] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Aug 31 09:35:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:56.419182 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a97176-9127-4de6-b091-eb460d1882a0 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:56.463504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f6af2-0733-4dd7-8ffd-04a07b2a780a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:56.501762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b755ff-274f-4343-881c-5a85150402f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:56.502768 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b755ff-274f-4343-881c-5a85150402f8 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:56.503221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b755ff-274f-4343-881c-5a85150402f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:56.503649 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b755ff-274f-4343-881c-5a85150402f8 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:56.504643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b755ff-274f-4343-881c-5a85150402f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:56.505177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b755ff-274f-4343-881c-5a85150402f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:56.506893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b755ff-274f-4343-881c-5a85150402f8 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:56.526461 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 439/867] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:56.529578 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b755ff-274f-4343-881c-5a85150402f8 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:56.531062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a97176-9127-4de6-b091-eb460d1882a0 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:56.567773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac726e-4fb7-4d4b-9dad-4a56b23910d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:56.568382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac726e-4fb7-4d4b-9dad-4a56b23910d8 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:56.568655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac726e-4fb7-4d4b-9dad-4a56b23910d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:56.568920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac726e-4fb7-4d4b-9dad-4a56b23910d8 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:56.570291 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac726e-4fb7-4d4b-9dad-4a56b23910d8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:56.570291 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac726e-4fb7-4d4b-9dad-4a56b23910d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:56.571221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac726e-4fb7-4d4b-9dad-4a56b23910d8 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:56.581523 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 429/868] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:56.582842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac726e-4fb7-4d4b-9dad-4a56b23910d8 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:56.598945 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:56.599931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:56.645212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b755ff-274f-4343-881c-5a85150402f8 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:56.682631 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d747b70-0960-4435-9f81-1eec6c466400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:56.683288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d747b70-0960-4435-9f81-1eec6c466400 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:56.683567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d747b70-0960-4435-9f81-1eec6c466400 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:56.684042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d747b70-0960-4435-9f81-1eec6c466400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:56.684562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d747b70-0960-4435-9f81-1eec6c466400 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=c5cf86ae159749cea0cc9783bffbfe75) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:56.684886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d747b70-0960-4435-9f81-1eec6c466400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:56.686284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d747b70-0960-4435-9f81-1eec6c466400 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:56.697929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac726e-4fb7-4d4b-9dad-4a56b23910d8 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:56.698330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 440/869] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:56 2022] GET /identity/v3/role_assignments?user.id=c5cf86ae159749cea0cc9783bffbfe75&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:56.741115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96418f7-1971-47ea-919a-2d222f0f0fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:56.742168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96418f7-1971-47ea-919a-2d222f0f0fcf None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:56.742624 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96418f7-1971-47ea-919a-2d222f0f0fcf None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:56.743061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96418f7-1971-47ea-919a-2d222f0f0fcf None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:56.743985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96418f7-1971-47ea-919a-2d222f0f0fcf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:56.744463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96418f7-1971-47ea-919a-2d222f0f0fcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:56.746145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96418f7-1971-47ea-919a-2d222f0f0fcf None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:56.759420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 430/870] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:56 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:57.754660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da7bdb-a4e4-463a-a233-f064e003c440 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:57.755224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da7bdb-a4e4-463a-a233-f064e003c440 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:57.755224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da7bdb-a4e4-463a-a233-f064e003c440 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:57.756838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 441/871] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:57.762697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c35026-adf6-4515-a541-36a6de7a5c1c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:57.763170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c35026-adf6-4515-a541-36a6de7a5c1c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:57.763443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c35026-adf6-4515-a541-36a6de7a5c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:57.783478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-c8c35026-adf6-4515-a541-36a6de7a5c1c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:57.784677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c8c35026-adf6-4515-a541-36a6de7a5c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:57.858169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 431/872] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:58.142039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da7bdb-a4e4-463a-a233-f064e003c440 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:58.231527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ebaf0-2205-4a51-819f-ae481dc71838 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:58.231756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ebaf0-2205-4a51-819f-ae481dc71838 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:58.231991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ebaf0-2205-4a51-819f-ae481dc71838 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:58.233571 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 432/873] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:58.238779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf06ad-609c-430e-a34d-909af2ac21ec None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:58.239059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf06ad-609c-430e-a34d-909af2ac21ec None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:58.239356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf06ad-609c-430e-a34d-909af2ac21ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:58.258146 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-1fdf06ad-609c-430e-a34d-909af2ac21ec None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:58.259904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1fdf06ad-609c-430e-a34d-909af2ac21ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:58.261593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da7bdb-a4e4-463a-a233-f064e003c440 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:58.262728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:58.289414 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:58.302953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c077ced-c105-4a52-bf1e-06eb89d3e607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:58.303619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c077ced-c105-4a52-bf1e-06eb89d3e607 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:58.303949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c077ced-c105-4a52-bf1e-06eb89d3e607 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:58.304222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c077ced-c105-4a52-bf1e-06eb89d3e607 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:58.315842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c077ced-c105-4a52-bf1e-06eb89d3e607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-6c077ced-c105-4a52-bf1e-06eb89d3e607 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:58.320590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:58.322098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:35:58.322955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 442/874] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Aug 31 09:35:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:58.328296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c077ced-c105-4a52-bf1e-06eb89d3e607 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:58.346431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 433/875] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:58.438205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c077ced-c105-4a52-bf1e-06eb89d3e607 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:58.474221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b8036-0cac-4633-8083-613588510be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:58.475240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b8036-0cac-4633-8083-613588510be3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:58.475715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b8036-0cac-4633-8083-613588510be3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:58.476204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b8036-0cac-4633-8083-613588510be3 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:58.477059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b8036-0cac-4633-8083-613588510be3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:58.477606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b8036-0cac-4633-8083-613588510be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:58.479926 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b8036-0cac-4633-8083-613588510be3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:58.488772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 443/876] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 09:35:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:58.496204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf06ad-609c-430e-a34d-909af2ac21ec None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:58.608235 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf06ad-609c-430e-a34d-909af2ac21ec None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:58.622143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b8036-0cac-4633-8083-613588510be3 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:58.652009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1833d-ad10-4a32-b120-0c8f4ee58b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:58.653070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1833d-ad10-4a32-b120-0c8f4ee58b45 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:58.653349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1833d-ad10-4a32-b120-0c8f4ee58b45 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:58.653881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1833d-ad10-4a32-b120-0c8f4ee58b45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:58.663571 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:58.671311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1833d-ad10-4a32-b120-0c8f4ee58b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:58.673036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1833d-ad10-4a32-b120-0c8f4ee58b45 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:58.674894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-50c1833d-ad10-4a32-b120-0c8f4ee58b45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:58.695048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:58.696088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:58.697396 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:58.697396 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:58.697396 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:58.697396 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 434/877] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:58.704162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1833d-ad10-4a32-b120-0c8f4ee58b45 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:58.736507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b8036-0cac-4633-8083-613588510be3 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:58.773070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e29646-ce5e-4b43-9f4e-1b5cf229c47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:58.773732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e29646-ce5e-4b43-9f4e-1b5cf229c47a None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:58.774011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e29646-ce5e-4b43-9f4e-1b5cf229c47a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:58.774289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e29646-ce5e-4b43-9f4e-1b5cf229c47a None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:58.775563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e29646-ce5e-4b43-9f4e-1b5cf229c47a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:58.776935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e29646-ce5e-4b43-9f4e-1b5cf229c47a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:58.799545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 444/878] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:35:58 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:35:58.801000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4518c9f-0be7-4733-b330-45392742af54 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:58.801685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4518c9f-0be7-4733-b330-45392742af54 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:58.801685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4518c9f-0be7-4733-b330-45392742af54 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:58.803186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 445/879] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:58.809048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750c155-7fe3-4752-8c70-e3429600f09d None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:58.809391 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750c155-7fe3-4752-8c70-e3429600f09d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:58.809591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750c155-7fe3-4752-8c70-e3429600f09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:58.822578 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1833d-ad10-4a32-b120-0c8f4ee58b45 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:58.830441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b750c155-7fe3-4752-8c70-e3429600f09d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:58.831733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b750c155-7fe3-4752-8c70-e3429600f09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:58.871964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc85b2f-de7c-4a22-8671-f3bdbeeeb3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:58.872628 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc85b2f-de7c-4a22-8671-f3bdbeeeb3b5 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:58.872974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc85b2f-de7c-4a22-8671-f3bdbeeeb3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:58.873310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc85b2f-de7c-4a22-8671-f3bdbeeeb3b5 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:58.874030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc85b2f-de7c-4a22-8671-f3bdbeeeb3b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:58.874398 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc85b2f-de7c-4a22-8671-f3bdbeeeb3b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:58.875841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc85b2f-de7c-4a22-8671-f3bdbeeeb3b5 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:58.894522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 435/880] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:58.894522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bf093-4765-4f01-b4e6-dae92491627f None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:58.894522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bf093-4765-4f01-b4e6-dae92491627f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:58.894522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bf093-4765-4f01-b4e6-dae92491627f None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:58.894522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 436/881] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:58.897930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bf093-4765-4f01-b4e6-dae92491627f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:58.923222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 446/882] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:58 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:58.923222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b62119-fb94-4fa3-85d3-0416c2d4aa96 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:58.923222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b62119-fb94-4fa3-85d3-0416c2d4aa96 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:58.923222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b62119-fb94-4fa3-85d3-0416c2d4aa96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:58.951871 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-43b62119-fb94-4fa3-85d3-0416c2d4aa96 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:58.953659 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-43b62119-fb94-4fa3-85d3-0416c2d4aa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:58.961642 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:58.992879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:59.036065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bf093-4765-4f01-b4e6-dae92491627f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:59.040752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 447/883] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:58 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:59.077856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915ba00-8e25-4c8a-8c3e-bc8f7ef93351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:59.079542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915ba00-8e25-4c8a-8c3e-bc8f7ef93351 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:59.079542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915ba00-8e25-4c8a-8c3e-bc8f7ef93351 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:59.079542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915ba00-8e25-4c8a-8c3e-bc8f7ef93351 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:59.087363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915ba00-8e25-4c8a-8c3e-bc8f7ef93351 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:59.088848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915ba00-8e25-4c8a-8c3e-bc8f7ef93351 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-2915ba00-8e25-4c8a-8c3e-bc8f7ef93351 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:59.101132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:59.102125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:59.103699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:59.103699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:59.103699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:59.103699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:59.103699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:59.103699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:59.103699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:59.103699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:59.103699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:59.103699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:59.103699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 437/884] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 09:35:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:59.108766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b62119-fb94-4fa3-85d3-0416c2d4aa96 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:59.210463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915ba00-8e25-4c8a-8c3e-bc8f7ef93351 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:59.220873 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b62119-fb94-4fa3-85d3-0416c2d4aa96 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:59.274298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08fa4d-e0d1-4667-8a17-0f5a51391837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:59.275892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08fa4d-e0d1-4667-8a17-0f5a51391837 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:59.276665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08fa4d-e0d1-4667-8a17-0f5a51391837 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:59.277136 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08fa4d-e0d1-4667-8a17-0f5a51391837 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:59.278383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08fa4d-e0d1-4667-8a17-0f5a51391837 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:59.278875 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08fa4d-e0d1-4667-8a17-0f5a51391837 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:59.281145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08fa4d-e0d1-4667-8a17-0f5a51391837 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:59.311329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 448/885] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Aug 31 09:35:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:59.312276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08fa4d-e0d1-4667-8a17-0f5a51391837 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:59.339815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915ba00-8e25-4c8a-8c3e-bc8f7ef93351 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:59.344712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:59.348271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:59.377105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb9e6b-a122-4450-ad7c-125ee596564d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:59.377718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb9e6b-a122-4450-ad7c-125ee596564d None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:59.377992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb9e6b-a122-4450-ad7c-125ee596564d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:59.378557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb9e6b-a122-4450-ad7c-125ee596564d None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:59.378924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb9e6b-a122-4450-ad7c-125ee596564d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:59.380324 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb9e6b-a122-4450-ad7c-125ee596564d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:59.398733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 438/886] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:35:59 2022] POST /identity/v3/services => generated 269 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:35:59.400619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb9e6b-a122-4450-ad7c-125ee596564d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:59.439641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08fa4d-e0d1-4667-8a17-0f5a51391837 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:59.504055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde65478-e7d7-46b1-8021-44cecac4e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:59.504712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde65478-e7d7-46b1-8021-44cecac4e088 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:59.504998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde65478-e7d7-46b1-8021-44cecac4e088 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:59.505323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde65478-e7d7-46b1-8021-44cecac4e088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:59.545063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb9e6b-a122-4450-ad7c-125ee596564d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:59.600073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493c469-a6bd-4846-b8a3-31a6ce86d5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:59.600716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493c469-a6bd-4846-b8a3-31a6ce86d5a0 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:59.601051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493c469-a6bd-4846-b8a3-31a6ce86d5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:59.601332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493c469-a6bd-4846-b8a3-31a6ce86d5a0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:59.601924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493c469-a6bd-4846-b8a3-31a6ce86d5a0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:59.603717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493c469-a6bd-4846-b8a3-31a6ce86d5a0 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:59.606209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde65478-e7d7-46b1-8021-44cecac4e088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:59.607711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde65478-e7d7-46b1-8021-44cecac4e088 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-9493c469-a6bd-4846-b8a3-31a6ce86d5a0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:59.614753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:59.616035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:59.616855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:35:59.620001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 439/887] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:59.626761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493c469-a6bd-4846-b8a3-31a6ce86d5a0 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:59.634742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:59.675197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:59.752043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 449/888] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:59 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:59.758678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde65478-e7d7-46b1-8021-44cecac4e088 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:59.773218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493c469-a6bd-4846-b8a3-31a6ce86d5a0 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:59.814435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e49d24-374d-460c-8d4b-31adabaf0432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:59.815525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e49d24-374d-460c-8d4b-31adabaf0432 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:59.816139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e49d24-374d-460c-8d4b-31adabaf0432 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:59.816605 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e49d24-374d-460c-8d4b-31adabaf0432 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:59.817542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e49d24-374d-460c-8d4b-31adabaf0432 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:59.818156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e49d24-374d-460c-8d4b-31adabaf0432 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:59.820245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e49d24-374d-460c-8d4b-31adabaf0432 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:59.827274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 440/889] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 09:35:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:59.832663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e49d24-374d-460c-8d4b-31adabaf0432 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:59.892919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde65478-e7d7-46b1-8021-44cecac4e088 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:59.923792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:59.948223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75976f-2b0c-433b-bdfa-737b0386311b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:59.949020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75976f-2b0c-433b-bdfa-737b0386311b None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:59.949306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75976f-2b0c-433b-bdfa-737b0386311b None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:59.949582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75976f-2b0c-433b-bdfa-737b0386311b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:59.967981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75976f-2b0c-433b-bdfa-737b0386311b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:59.969612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75976f-2b0c-433b-bdfa-737b0386311b None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:59.980619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:59.990468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e49d24-374d-460c-8d4b-31adabaf0432 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-fe75976f-2b0c-433b-bdfa-737b0386311b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:59.997444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:59.998413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:59.999777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:59.999777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:59.999777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:00.003257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 450/890] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:00.006935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75976f-2b0c-433b-bdfa-737b0386311b None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:00.050731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e47416-2766-449f-82cd-98b754586775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:00.052527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e47416-2766-449f-82cd-98b754586775 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:00.053204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e47416-2766-449f-82cd-98b754586775 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:00.054547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e47416-2766-449f-82cd-98b754586775 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:00.056347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e47416-2766-449f-82cd-98b754586775 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:00.057084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e47416-2766-449f-82cd-98b754586775 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:00.059730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e47416-2766-449f-82cd-98b754586775 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:00.073726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 441/891] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 09:35:59 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:00.208788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75976f-2b0c-433b-bdfa-737b0386311b None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:00.281202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c1981-ba35-4936-bcc9-d67ecaeea00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:00.281846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1981-ba35-4936-bcc9-d67ecaeea00d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:00.282130 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1981-ba35-4936-bcc9-d67ecaeea00d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:00.282406 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1981-ba35-4936-bcc9-d67ecaeea00d None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:00.283083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c1981-ba35-4936-bcc9-d67ecaeea00d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:00.283421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c1981-ba35-4936-bcc9-d67ecaeea00d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:00.284886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c1981-ba35-4936-bcc9-d67ecaeea00d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:00.309447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 451/892] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:00.315722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e47416-2766-449f-82cd-98b754586775 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:00.434709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e47416-2766-449f-82cd-98b754586775 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:00.490128 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ce56d-ee1c-49a3-8e14-92d2e344ed49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:00.490774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ce56d-ee1c-49a3-8e14-92d2e344ed49 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:00.491051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ce56d-ee1c-49a3-8e14-92d2e344ed49 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:00.491893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ce56d-ee1c-49a3-8e14-92d2e344ed49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:00.508283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ce56d-ee1c-49a3-8e14-92d2e344ed49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:00.510663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ce56d-ee1c-49a3-8e14-92d2e344ed49 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-002ce56d-ee1c-49a3-8e14-92d2e344ed49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:00.524317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:00.525364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:00.526271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:00.526271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:00.526271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:00.526271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:00.526271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:00.526271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:00.526271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:00.526271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:00.526271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 442/893] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:00.536325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c1981-ba35-4936-bcc9-d67ecaeea00d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:00.597041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:00.686737 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c1981-ba35-4936-bcc9-d67ecaeea00d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:00.723409 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d087b-370a-457d-85ba-50f50a822641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:00.724020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d087b-370a-457d-85ba-50f50a822641 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:00.724300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d087b-370a-457d-85ba-50f50a822641 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:00.724575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d087b-370a-457d-85ba-50f50a822641 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:00.725275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d087b-370a-457d-85ba-50f50a822641 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:00.725581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d087b-370a-457d-85ba-50f50a822641 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:00.727013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d087b-370a-457d-85ba-50f50a822641 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:00.739728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 452/894] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:36:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:00.746296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ce56d-ee1c-49a3-8e14-92d2e344ed49 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:00.767529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:00.854807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ce56d-ee1c-49a3-8e14-92d2e344ed49 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:00.877993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a47987-633d-4ca4-97b5-91b06299b3ea None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:00.878455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a47987-633d-4ca4-97b5-91b06299b3ea None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:00.878805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a47987-633d-4ca4-97b5-91b06299b3ea None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:00.880499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 453/895] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:00.886112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd5562-64e8-4090-978b-98f024da14fb None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:00.886398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd5562-64e8-4090-978b-98f024da14fb None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:00.886671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd5562-64e8-4090-978b-98f024da14fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:00.893556 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60275c28-e072-4d37-8cbf-1b08027dfc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:00.895310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60275c28-e072-4d37-8cbf-1b08027dfc0b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:00.895735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60275c28-e072-4d37-8cbf-1b08027dfc0b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:00.896171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60275c28-e072-4d37-8cbf-1b08027dfc0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:00.897063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60275c28-e072-4d37-8cbf-1b08027dfc0b None admin] RBAC: Adding query filter params (role.id=5bb2daf5cef542028e8802f2f06a46bd, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=0a2e5c1014f1441aa47d5de46462b6d9) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:00.897514 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60275c28-e072-4d37-8cbf-1b08027dfc0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:00.899092 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60275c28-e072-4d37-8cbf-1b08027dfc0b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:00.906564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-6cfd5562-64e8-4090-978b-98f024da14fb None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:00.907587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 443/896] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:00 2022] GET /identity/v3/role_assignments?user.id=0a2e5c1014f1441aa47d5de46462b6d9&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=5bb2daf5cef542028e8802f2f06a46bd => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:00.911453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6cfd5562-64e8-4090-978b-98f024da14fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:00.989440 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 454/897] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:01.267425 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60275c28-e072-4d37-8cbf-1b08027dfc0b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:01.359686 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:01.383609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60275c28-e072-4d37-8cbf-1b08027dfc0b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:01.422742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e2ca6-cbab-40a8-aa5d-df39b84c8a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:01.423812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e2ca6-cbab-40a8-aa5d-df39b84c8a6f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:01.424323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e2ca6-cbab-40a8-aa5d-df39b84c8a6f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:01.424797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e2ca6-cbab-40a8-aa5d-df39b84c8a6f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:01.425578 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e2ca6-cbab-40a8-aa5d-df39b84c8a6f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:01.427200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e2ca6-cbab-40a8-aa5d-df39b84c8a6f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-883e2ca6-cbab-40a8-aa5d-df39b84c8a6f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:01.437307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:01.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:36:01.439168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:01.441939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 444/898] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:36:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:01.447567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd5562-64e8-4090-978b-98f024da14fb None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:01.476608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:01.526620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa5ca6-8a1b-49e3-91cb-55f35233371b None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:01.526868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa5ca6-8a1b-49e3-91cb-55f35233371b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:01.527096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa5ca6-8a1b-49e3-91cb-55f35233371b None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:01.529738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 445/899] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:01.535715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304b88c-7a6c-4c6a-b146-14c8ffb84695 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:01.536114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304b88c-7a6c-4c6a-b146-14c8ffb84695 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:01.536490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304b88c-7a6c-4c6a-b146-14c8ffb84695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:01.558352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-8304b88c-7a6c-4c6a-b146-14c8ffb84695 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:01.561347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-8304b88c-7a6c-4c6a-b146-14c8ffb84695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:01.562952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd5562-64e8-4090-978b-98f024da14fb None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:01.604749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca40371-37a1-46b0-be45-062d6c42f0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:01.605692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca40371-37a1-46b0-be45-062d6c42f0b2 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:01.606118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca40371-37a1-46b0-be45-062d6c42f0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:01.606562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca40371-37a1-46b0-be45-062d6c42f0b2 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:01.607377 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca40371-37a1-46b0-be45-062d6c42f0b2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:01.607824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca40371-37a1-46b0-be45-062d6c42f0b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:01.609360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca40371-37a1-46b0-be45-062d6c42f0b2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:01.617513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 455/900] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:36:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:01.623251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca40371-37a1-46b0-be45-062d6c42f0b2 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:01.653629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 446/901] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:01 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:01.727644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:01.737373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca40371-37a1-46b0-be45-062d6c42f0b2 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:01.774173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480de518-9197-43ab-939b-4817d060c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:01.775453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480de518-9197-43ab-939b-4817d060c780 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:01.775745 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480de518-9197-43ab-939b-4817d060c780 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:01.776009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480de518-9197-43ab-939b-4817d060c780 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:01.776767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480de518-9197-43ab-939b-4817d060c780 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:01.777016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480de518-9197-43ab-939b-4817d060c780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:01.778385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480de518-9197-43ab-939b-4817d060c780 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:01.783956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 456/902] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:36:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:01.792603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304b88c-7a6c-4c6a-b146-14c8ffb84695 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:01.876964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:01.903630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304b88c-7a6c-4c6a-b146-14c8ffb84695 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:01.939242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03316b-a312-478e-9aa4-9917b1b5e0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:01.939852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03316b-a312-478e-9aa4-9917b1b5e0c5 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:01.940126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03316b-a312-478e-9aa4-9917b1b5e0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:01.940394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03316b-a312-478e-9aa4-9917b1b5e0c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:01.940716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480de518-9197-43ab-939b-4817d060c780 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:01.941160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03316b-a312-478e-9aa4-9917b1b5e0c5 None admin] RBAC: Adding query filter params (interface=public, service_id=cace5771d04f40f5a0390bce89c859c6, region_id=RegionOne) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:01.941386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03316b-a312-478e-9aa4-9917b1b5e0c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:01.942767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03316b-a312-478e-9aa4-9917b1b5e0c5 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:01.949526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 447/903] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 09:36:01 2022] GET /identity/v3/endpoints?service_id=cace5771d04f40f5a0390bce89c859c6&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:01.956158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03316b-a312-478e-9aa4-9917b1b5e0c5 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:02.055215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480de518-9197-43ab-939b-4817d060c780 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:02.068419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03316b-a312-478e-9aa4-9917b1b5e0c5 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:02.107111 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a37e96-c2a8-4697-b383-1cbc2cbf2d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:02.108061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a37e96-c2a8-4697-b383-1cbc2cbf2d6f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.108746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a37e96-c2a8-4697-b383-1cbc2cbf2d6f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.109188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a37e96-c2a8-4697-b383-1cbc2cbf2d6f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.110261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a37e96-c2a8-4697-b383-1cbc2cbf2d6f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:02.112574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a37e96-c2a8-4697-b383-1cbc2cbf2d6f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:02.123021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609fad6-b700-45f4-b367-99c556db941f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:02.123778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609fad6-b700-45f4-b367-99c556db941f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.124148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609fad6-b700-45f4-b367-99c556db941f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.124512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609fad6-b700-45f4-b367-99c556db941f None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-b1a37e96-c2a8-4697-b383-1cbc2cbf2d6f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:02.124929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:02.126014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:36:02.126951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:02.127744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609fad6-b700-45f4-b367-99c556db941f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:02.127823 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609fad6-b700-45f4-b367-99c556db941f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:02.129923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 457/904] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Aug 31 09:36:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:02.138535 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a37e96-c2a8-4697-b383-1cbc2cbf2d6f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:02.143341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 448/905] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 09:36:01 2022] GET /identity/v3/services => generated 1268 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:36:02.159751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:02.263705 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a37e96-c2a8-4697-b383-1cbc2cbf2d6f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:02.306311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a972b-8d23-4041-a90f-e52cf1806a69 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.306931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a972b-8d23-4041-a90f-e52cf1806a69 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.307465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a972b-8d23-4041-a90f-e52cf1806a69 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.309194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d3150-b7d5-4127-8407-d52cc127db07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:02.313679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d3150-b7d5-4127-8407-d52cc127db07 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.313679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d3150-b7d5-4127-8407-d52cc127db07 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.313679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d3150-b7d5-4127-8407-d52cc127db07 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.313679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d3150-b7d5-4127-8407-d52cc127db07 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:02.313679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d3150-b7d5-4127-8407-d52cc127db07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:02.313679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d3150-b7d5-4127-8407-d52cc127db07 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:02.314421 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 449/906] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:36:02.316488 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c2f30-587e-427d-b0b8-3ec8b574d3db None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.317026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c2f30-587e-427d-b0b8-3ec8b574d3db None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.317512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c2f30-587e-427d-b0b8-3ec8b574d3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.318824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 458/907] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 09:36:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:02.328614 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:02.332213 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d3150-b7d5-4127-8407-d52cc127db07 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:02.341099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-879c2f30-587e-427d-b0b8-3ec8b574d3db None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:02.342308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-879c2f30-587e-427d-b0b8-3ec8b574d3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:02.424653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 450/908] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:02.434882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:02.461428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d3150-b7d5-4127-8407-d52cc127db07 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:02.498761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb9c95-efc0-44ab-9146-aaf2ab8eea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:02.499756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb9c95-efc0-44ab-9146-aaf2ab8eea6f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.500258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb9c95-efc0-44ab-9146-aaf2ab8eea6f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.500722 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb9c95-efc0-44ab-9146-aaf2ab8eea6f None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.501541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb9c95-efc0-44ab-9146-aaf2ab8eea6f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:02.502034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb9c95-efc0-44ab-9146-aaf2ab8eea6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:02.503608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb9c95-efc0-44ab-9146-aaf2ab8eea6f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:02.509326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 459/909] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 09:36:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:02.515839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c2f30-587e-427d-b0b8-3ec8b574d3db None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:02.631481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c2f30-587e-427d-b0b8-3ec8b574d3db None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:02.668506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7f4de-34f6-4562-8397-8319e04c161a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:02.669115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7f4de-34f6-4562-8397-8319e04c161a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.669393 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7f4de-34f6-4562-8397-8319e04c161a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.669668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7f4de-34f6-4562-8397-8319e04c161a None admin] PATH_INFO: `/v3/endpoints` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.670374 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7f4de-34f6-4562-8397-8319e04c161a None admin] RBAC: Adding query filter params (interface=public, service_id=442ca9df4f5c4bb9984cb83f0f081335, region_id=RegionOne) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:02.670684 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7f4de-34f6-4562-8397-8319e04c161a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:02.672095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7f4de-34f6-4562-8397-8319e04c161a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:02.673816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:02.678785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 451/910] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 09:36:02 2022] GET /identity/v3/endpoints?service_id=442ca9df4f5c4bb9984cb83f0f081335&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:02.684514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb9c95-efc0-44ab-9146-aaf2ab8eea6f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:02.738014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7f4de-34f6-4562-8397-8319e04c161a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:02.799624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb9c95-efc0-44ab-9146-aaf2ab8eea6f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:02.836642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe4d65-23a9-4be7-b119-d548dc31aa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:02.837335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe4d65-23a9-4be7-b119-d548dc31aa4e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.837618 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe4d65-23a9-4be7-b119-d548dc31aa4e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.837899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe4d65-23a9-4be7-b119-d548dc31aa4e None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.838529 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe4d65-23a9-4be7-b119-d548dc31aa4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:02.839955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe4d65-23a9-4be7-b119-d548dc31aa4e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:02.846544 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 460/911] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 09:36:02 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:36:02.856321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7f4de-34f6-4562-8397-8319e04c161a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:02.899953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad646dd6-4a1c-4256-afda-f53c14f0d54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:02.900626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad646dd6-4a1c-4256-afda-f53c14f0d54d None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.900860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad646dd6-4a1c-4256-afda-f53c14f0d54d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.901130 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad646dd6-4a1c-4256-afda-f53c14f0d54d None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.901811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad646dd6-4a1c-4256-afda-f53c14f0d54d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:02.903277 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad646dd6-4a1c-4256-afda-f53c14f0d54d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:02.931369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 452/912] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:36:02 2022] POST /identity/v3/services => generated 241 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:36:02.943208 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb137709-6c0f-4eb6-92c8-71c55f6d9d74 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.943751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb137709-6c0f-4eb6-92c8-71c55f6d9d74 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.943751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb137709-6c0f-4eb6-92c8-71c55f6d9d74 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.945511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 461/913] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:02.951572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97213d0-84e1-49f6-8c8d-8864d0f77627 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.951922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97213d0-84e1-49f6-8c8d-8864d0f77627 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.952141 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97213d0-84e1-49f6-8c8d-8864d0f77627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.973615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a97213d0-84e1-49f6-8c8d-8864d0f77627 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:02.975018 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a97213d0-84e1-49f6-8c8d-8864d0f77627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:03.050051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:03.083751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 453/914] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:03.357855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb137709-6c0f-4eb6-92c8-71c55f6d9d74 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:03.367587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:03.476723 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb137709-6c0f-4eb6-92c8-71c55f6d9d74 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:03.514107 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695c713-6cb2-4aef-9aa8-2797c72b0502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:03.515011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695c713-6cb2-4aef-9aa8-2797c72b0502 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:03.515325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695c713-6cb2-4aef-9aa8-2797c72b0502 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:03.515602 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695c713-6cb2-4aef-9aa8-2797c72b0502 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:03.527425 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695c713-6cb2-4aef-9aa8-2797c72b0502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-5695c713-6cb2-4aef-9aa8-2797c72b0502 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:03.532278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:03.533194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:03.534209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 462/915] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Aug 31 09:36:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:03.539412 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97213d0-84e1-49f6-8c8d-8864d0f77627 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:03.649392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97213d0-84e1-49f6-8c8d-8864d0f77627 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:03.690977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2dede4-44ca-4601-b008-758171d8f0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:03.691650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2dede4-44ca-4601-b008-758171d8f0b7 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:03.691925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2dede4-44ca-4601-b008-758171d8f0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:03.692193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2dede4-44ca-4601-b008-758171d8f0b7 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:03.692874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2dede4-44ca-4601-b008-758171d8f0b7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:03.693173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2dede4-44ca-4601-b008-758171d8f0b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:03.694589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2dede4-44ca-4601-b008-758171d8f0b7 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:03.704631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 454/916] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 09:36:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:03.710419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695c713-6cb2-4aef-9aa8-2797c72b0502 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:03.792585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:03.831208 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695c713-6cb2-4aef-9aa8-2797c72b0502 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:03.869413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ad203-22e3-49d2-81a8-c752ea2e828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:03.870056 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ad203-22e3-49d2-81a8-c752ea2e828d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:03.870338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ad203-22e3-49d2-81a8-c752ea2e828d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:03.870613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ad203-22e3-49d2-81a8-c752ea2e828d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:03.882127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ad203-22e3-49d2-81a8-c752ea2e828d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:03.884483 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ad203-22e3-49d2-81a8-c752ea2e828d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-b76ad203-22e3-49d2-81a8-c752ea2e828d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:03.900805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:03.901748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:03.902939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:03.902939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:03.902939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:03.902939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 463/917] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:03.908203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2dede4-44ca-4601-b008-758171d8f0b7 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:03.965743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:04.023735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2dede4-44ca-4601-b008-758171d8f0b7 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:04.064206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd2ae3-5352-4f58-b1c4-4b289afe73b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:04.064828 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd2ae3-5352-4f58-b1c4-4b289afe73b6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:04.065103 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd2ae3-5352-4f58-b1c4-4b289afe73b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:04.065372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd2ae3-5352-4f58-b1c4-4b289afe73b6 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:04.066022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd2ae3-5352-4f58-b1c4-4b289afe73b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:04.066320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd2ae3-5352-4f58-b1c4-4b289afe73b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:04.067748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd2ae3-5352-4f58-b1c4-4b289afe73b6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:04.081673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 455/918] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:04.087923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ad203-22e3-49d2-81a8-c752ea2e828d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:04.201017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ad203-22e3-49d2-81a8-c752ea2e828d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:04.226483 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:04.238186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a940e97-2808-4fe2-b7cf-1649586ee35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:04.239086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a940e97-2808-4fe2-b7cf-1649586ee35d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:04.239635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a940e97-2808-4fe2-b7cf-1649586ee35d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:04.240057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a940e97-2808-4fe2-b7cf-1649586ee35d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:04.248399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a940e97-2808-4fe2-b7cf-1649586ee35d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:04.250014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a940e97-2808-4fe2-b7cf-1649586ee35d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-7a940e97-2808-4fe2-b7cf-1649586ee35d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:36:04.262235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:04.263424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:04.264855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:04.264855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:04.264855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:36:04.264855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:36:04.264855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:36:04.264855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:36:04.264855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:36:04.264855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:36:04.264855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:36:04.264855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:04.267015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 464/919] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 09:36:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:04.278221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd2ae3-5352-4f58-b1c4-4b289afe73b6 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:04.298094 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3296c00-bc09-4455-9461-550e14bfcfe7 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:04.298280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3296c00-bc09-4455-9461-550e14bfcfe7 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:04.298526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3296c00-bc09-4455-9461-550e14bfcfe7 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:04.300164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 465/920] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:36:04.305095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b151-d296-42f1-8cb9-79b12a6b2310 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:04.305384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b151-d296-42f1-8cb9-79b12a6b2310 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:04.305767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b151-d296-42f1-8cb9-79b12a6b2310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:04.325953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-aee0b151-d296-42f1-8cb9-79b12a6b2310 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:04.327275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-aee0b151-d296-42f1-8cb9-79b12a6b2310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:04.399297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd2ae3-5352-4f58-b1c4-4b289afe73b6 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:04.410863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:04.413816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 466/921] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:04.444822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005fef4-942f-47b0-88bd-c96d5d050208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:04.445757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005fef4-942f-47b0-88bd-c96d5d050208 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:04.446299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005fef4-942f-47b0-88bd-c96d5d050208 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:04.446848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005fef4-942f-47b0-88bd-c96d5d050208 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:04.447760 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005fef4-942f-47b0-88bd-c96d5d050208 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:04.448214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005fef4-942f-47b0-88bd-c96d5d050208 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:04.449806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005fef4-942f-47b0-88bd-c96d5d050208 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:04.469459 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 456/922] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Aug 31 09:36:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:04.475283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0b151-d296-42f1-8cb9-79b12a6b2310 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:04.584802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0b151-d296-42f1-8cb9-79b12a6b2310 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:04.596038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:04.625165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5ff30-349c-4114-816b-94ff9382b965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:04.625812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5ff30-349c-4114-816b-94ff9382b965 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:04.626096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5ff30-349c-4114-816b-94ff9382b965 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:04.626369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5ff30-349c-4114-816b-94ff9382b965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:04.638460 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5ff30-349c-4114-816b-94ff9382b965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:04.640212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5ff30-349c-4114-816b-94ff9382b965 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-78b5ff30-349c-4114-816b-94ff9382b965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:04.657069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:04.658093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:04.658989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:04.658989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:04.658989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:04.658989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 467/923] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:04.666372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005fef4-942f-47b0-88bd-c96d5d050208 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:04.683133 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5ff30-349c-4114-816b-94ff9382b965 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:04.778685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005fef4-942f-47b0-88bd-c96d5d050208 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:04.796923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5ff30-349c-4114-816b-94ff9382b965 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:04.817109 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4945b-084b-40d6-8724-f849d057d36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:04.818151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4945b-084b-40d6-8724-f849d057d36e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:04.818631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4945b-084b-40d6-8724-f849d057d36e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:04.819098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4945b-084b-40d6-8724-f849d057d36e None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:04.819963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4945b-084b-40d6-8724-f849d057d36e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:04.820460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4945b-084b-40d6-8724-f849d057d36e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:04.822585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4945b-084b-40d6-8724-f849d057d36e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:04.834334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac97b16-4025-436e-9cfe-f73a1ba00d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:04.834972 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac97b16-4025-436e-9cfe-f73a1ba00d6e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:04.835282 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac97b16-4025-436e-9cfe-f73a1ba00d6e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:04.835559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac97b16-4025-436e-9cfe-f73a1ba00d6e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:04.836147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac97b16-4025-436e-9cfe-f73a1ba00d6e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:04.841380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac97b16-4025-436e-9cfe-f73a1ba00d6e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:04.841679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 457/924] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:04.842825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4945b-084b-40d6-8724-f849d057d36e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-bac97b16-4025-436e-9cfe-f73a1ba00d6e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:04.848298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:04.849267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:04.850089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 468/925] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:36:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:04.855834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac97b16-4025-436e-9cfe-f73a1ba00d6e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:04.875086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:04.883184 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:04.956626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4945b-084b-40d6-8724-f849d057d36e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:04.973713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac97b16-4025-436e-9cfe-f73a1ba00d6e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:04.995658 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b663466-3124-4fba-a807-1362d7cc0664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:04.996296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b663466-3124-4fba-a807-1362d7cc0664 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:04.996573 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b663466-3124-4fba-a807-1362d7cc0664 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:04.996844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b663466-3124-4fba-a807-1362d7cc0664 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.004656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b663466-3124-4fba-a807-1362d7cc0664 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:05.006135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b663466-3124-4fba-a807-1362d7cc0664 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:05.016378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a1116-b3e3-4969-96ae-b1622ca74972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:05.017283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a1116-b3e3-4969-96ae-b1622ca74972 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.017744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a1116-b3e3-4969-96ae-b1622ca74972 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.018203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a1116-b3e3-4969-96ae-b1622ca74972 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.019012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a1116-b3e3-4969-96ae-b1622ca74972 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-9b663466-3124-4fba-a807-1362d7cc0664 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:05.019192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:05.020037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:05.020915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:05.020915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:05.020915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:05.020915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:05.020915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:05.020915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:05.020915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:05.020915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:05.020915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 458/926] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:05.021753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31736ed1-0de8-4dc6-b414-c25d1ac651e8 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.021753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31736ed1-0de8-4dc6-b414-c25d1ac651e8 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.021960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31736ed1-0de8-4dc6-b414-c25d1ac651e8 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.022421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a1116-b3e3-4969-96ae-b1622ca74972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:05.023445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 459/927] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:05.024270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a1116-b3e3-4969-96ae-b1622ca74972 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:05.026392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31736ed1-0de8-4dc6-b414-c25d1ac651e8 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:05.033240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 469/928] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:36:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:05.034500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127ed5d-ad2f-465b-87ef-d8daac37389f None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.034789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127ed5d-ad2f-465b-87ef-d8daac37389f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.035058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127ed5d-ad2f-465b-87ef-d8daac37389f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.054858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0127ed5d-ad2f-465b-87ef-d8daac37389f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:05.056138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0127ed5d-ad2f-465b-87ef-d8daac37389f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:05.122717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:05.126884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 470/929] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:05.127775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127ed5d-ad2f-465b-87ef-d8daac37389f None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:05.141695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31736ed1-0de8-4dc6-b414-c25d1ac651e8 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:05.146393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:05.180434 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ca874-376e-43ee-ac0e-1d91a81684a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:05.182199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ca874-376e-43ee-ac0e-1d91a81684a1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.182474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ca874-376e-43ee-ac0e-1d91a81684a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.182744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ca874-376e-43ee-ac0e-1d91a81684a1 None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.183477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ca874-376e-43ee-ac0e-1d91a81684a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:05.183789 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ca874-376e-43ee-ac0e-1d91a81684a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:05.185205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ca874-376e-43ee-ac0e-1d91a81684a1 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:05.198101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 460/930] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:36:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:05.204297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d696a-b5e4-4b66-a992-702a1c095808 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.204567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d696a-b5e4-4b66-a992-702a1c095808 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.204853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d696a-b5e4-4b66-a992-702a1c095808 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.206188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 461/931] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:05.206885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d696a-b5e4-4b66-a992-702a1c095808 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:05.252911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127ed5d-ad2f-465b-87ef-d8daac37389f None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:05.290580 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0243d-7811-446a-834b-3079157239a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:05.291512 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0243d-7811-446a-834b-3079157239a5 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.291979 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0243d-7811-446a-834b-3079157239a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.292437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0243d-7811-446a-834b-3079157239a5 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.293077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0243d-7811-446a-834b-3079157239a5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:05.293381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0243d-7811-446a-834b-3079157239a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:05.294762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0243d-7811-446a-834b-3079157239a5 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:05.304248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 471/932] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 09:36:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:05.304248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a4ca8-fedf-4f0f-9335-ac4d64802569 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.304248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a4ca8-fedf-4f0f-9335-ac4d64802569 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.304248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a4ca8-fedf-4f0f-9335-ac4d64802569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.320081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-a72a4ca8-fedf-4f0f-9335-ac4d64802569 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:05.321950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a72a4ca8-fedf-4f0f-9335-ac4d64802569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:05.325705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d696a-b5e4-4b66-a992-702a1c095808 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:05.363489 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7435193-259d-4cef-80d8-3525f914d221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:05.364180 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7435193-259d-4cef-80d8-3525f914d221 None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.364450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7435193-259d-4cef-80d8-3525f914d221 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.364717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7435193-259d-4cef-80d8-3525f914d221 None admin] PATH_INFO: `/v3/projects/8ef8b586382941408cd88ee654c42d32/users/0a2e5c1014f1441aa47d5de46462b6d9/roles/5bb2daf5cef542028e8802f2f06a46bd` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.366953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7435193-259d-4cef-80d8-3525f914d221 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef8b586382941408cd88ee654c42d32, user_id=0a2e5c1014f1441aa47d5de46462b6d9, role_id=5bb2daf5cef542028e8802f2f06a46bd)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:05.369036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7435193-259d-4cef-80d8-3525f914d221 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:05.394745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 462/933] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:36:05 2022] PUT /identity/v3/projects/8ef8b586382941408cd88ee654c42d32/users/0a2e5c1014f1441aa47d5de46462b6d9/roles/5bb2daf5cef542028e8802f2f06a46bd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:36:05.394745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7435193-259d-4cef-80d8-3525f914d221 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:05.401038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 472/934] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:05.401038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a4ca8-fedf-4f0f-9335-ac4d64802569 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:05.401728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:05.419527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:05.548929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a4ca8-fedf-4f0f-9335-ac4d64802569 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:05.563732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7435193-259d-4cef-80d8-3525f914d221 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:05.600294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778e6b4-af55-46fb-86fa-c688fd821049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:05.601310 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778e6b4-af55-46fb-86fa-c688fd821049 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.601803 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778e6b4-af55-46fb-86fa-c688fd821049 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.602276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778e6b4-af55-46fb-86fa-c688fd821049 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.603155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778e6b4-af55-46fb-86fa-c688fd821049 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:05.604742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778e6b4-af55-46fb-86fa-c688fd821049 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:05.605517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124b04a-3d42-491d-84a1-95f113c51697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:05.606133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124b04a-3d42-491d-84a1-95f113c51697 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.606411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124b04a-3d42-491d-84a1-95f113c51697 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.606681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124b04a-3d42-491d-84a1-95f113c51697 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.607530 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124b04a-3d42-491d-84a1-95f113c51697 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:05.609262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124b04a-3d42-491d-84a1-95f113c51697 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-4778e6b4-af55-46fb-86fa-c688fd821049 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:05.616103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:05.617066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:36:05.617938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:05.621052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 473/935] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Aug 31 09:36:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:05.627895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778e6b4-af55-46fb-86fa-c688fd821049 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:05.634875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 463/936] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:36:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:05.679408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124b04a-3d42-491d-84a1-95f113c51697 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:05.748019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:05.779564 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:05.792451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778e6b4-af55-46fb-86fa-c688fd821049 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:05.842668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124b04a-3d42-491d-84a1-95f113c51697 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:05.855928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcc1d5-83d3-4bb4-a046-d3f542821e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:05.856552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcc1d5-83d3-4bb4-a046-d3f542821e21 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.856831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcc1d5-83d3-4bb4-a046-d3f542821e21 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.857102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcc1d5-83d3-4bb4-a046-d3f542821e21 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.857741 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcc1d5-83d3-4bb4-a046-d3f542821e21 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:05.858059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcc1d5-83d3-4bb4-a046-d3f542821e21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:05.860587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcc1d5-83d3-4bb4-a046-d3f542821e21 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:05.865585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 474/937] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 09:36:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:05.871582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcc1d5-83d3-4bb4-a046-d3f542821e21 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:05.902724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2760938-f9be-45fd-b0b5-7a78b9880653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:05.903395 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2760938-f9be-45fd-b0b5-7a78b9880653 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.903714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2760938-f9be-45fd-b0b5-7a78b9880653 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.903955 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2760938-f9be-45fd-b0b5-7a78b9880653 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.904527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2760938-f9be-45fd-b0b5-7a78b9880653 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:05.905899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2760938-f9be-45fd-b0b5-7a78b9880653 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-d2760938-f9be-45fd-b0b5-7a78b9880653 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:05.915331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:05.916391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:05.922200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 464/938] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 09:36:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:05.924257 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2760938-f9be-45fd-b0b5-7a78b9880653 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:06.006619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcc1d5-83d3-4bb4-a046-d3f542821e21 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:06.046839 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4924f5c-88e5-4671-8f7e-9eebd547adeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:06.047580 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4924f5c-88e5-4671-8f7e-9eebd547adeb None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:06.047926 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4924f5c-88e5-4671-8f7e-9eebd547adeb None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:06.048264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4924f5c-88e5-4671-8f7e-9eebd547adeb None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:06.048966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4924f5c-88e5-4671-8f7e-9eebd547adeb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:06.049334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4924f5c-88e5-4671-8f7e-9eebd547adeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:06.049825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2760938-f9be-45fd-b0b5-7a78b9880653 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:06.050838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4924f5c-88e5-4671-8f7e-9eebd547adeb None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:06.057842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 475/939] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 09:36:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:06.068592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4924f5c-88e5-4671-8f7e-9eebd547adeb None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:06.076758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:06.084415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:06.102668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43bc84-0c1b-4b4c-9bf4-e5707c8fe3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:06.103721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43bc84-0c1b-4b4c-9bf4-e5707c8fe3da None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:06.104223 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43bc84-0c1b-4b4c-9bf4-e5707c8fe3da None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:06.104859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43bc84-0c1b-4b4c-9bf4-e5707c8fe3da None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:06.106378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43bc84-0c1b-4b4c-9bf4-e5707c8fe3da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:06.107155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43bc84-0c1b-4b4c-9bf4-e5707c8fe3da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:06.108757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43bc84-0c1b-4b4c-9bf4-e5707c8fe3da None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:06.116137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 465/940] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 09:36:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:06.127179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43bc84-0c1b-4b4c-9bf4-e5707c8fe3da None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:06.208897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4924f5c-88e5-4671-8f7e-9eebd547adeb None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:06.253474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43bc84-0c1b-4b4c-9bf4-e5707c8fe3da None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:06.265214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4f234-a96f-4f50-9588-790e0bc856de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:06.265830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4f234-a96f-4f50-9588-790e0bc856de None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:06.266107 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4f234-a96f-4f50-9588-790e0bc856de None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:06.266628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4f234-a96f-4f50-9588-790e0bc856de None admin] PATH_INFO: `/v3/endpoints` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:06.267047 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4f234-a96f-4f50-9588-790e0bc856de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:06.268561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4f234-a96f-4f50-9588-790e0bc856de None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:06.299357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 476/941] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:36:06 2022] POST /identity/v3/endpoints => generated 347 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:06.310774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110b89f-beb7-4d94-aa59-fd73c145e9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:06.311922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110b89f-beb7-4d94-aa59-fd73c145e9cb None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:06.312502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110b89f-beb7-4d94-aa59-fd73c145e9cb None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:06.313165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110b89f-beb7-4d94-aa59-fd73c145e9cb None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:06.314354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110b89f-beb7-4d94-aa59-fd73c145e9cb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:06.314991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110b89f-beb7-4d94-aa59-fd73c145e9cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:06.317130 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110b89f-beb7-4d94-aa59-fd73c145e9cb None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:06.325370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 466/942] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 09:36:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:06.335160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4f234-a96f-4f50-9588-790e0bc856de None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:06.413535 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:06.468696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4f234-a96f-4f50-9588-790e0bc856de None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:06.512644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95a1c1-5844-48b2-ac70-6b82bd9c3c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:06.513282 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95a1c1-5844-48b2-ac70-6b82bd9c3c2e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:06.513558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95a1c1-5844-48b2-ac70-6b82bd9c3c2e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:06.513832 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95a1c1-5844-48b2-ac70-6b82bd9c3c2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:06.515447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95a1c1-5844-48b2-ac70-6b82bd9c3c2e None admin] RBAC: Adding query filter params (interface=public, service_id=978f8ee4897346f2b744454d818ae195, region_id=RegionOne) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:06.515447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95a1c1-5844-48b2-ac70-6b82bd9c3c2e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:06.516874 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95a1c1-5844-48b2-ac70-6b82bd9c3c2e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:06.523157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 477/943] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 09:36:06 2022] GET /identity/v3/endpoints?service_id=978f8ee4897346f2b744454d818ae195&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:06.529717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110b89f-beb7-4d94-aa59-fd73c145e9cb None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:06.582062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:06.653407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110b89f-beb7-4d94-aa59-fd73c145e9cb None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:06.689346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8eea90-b762-4163-9548-a7432e27fb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:06.690003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8eea90-b762-4163-9548-a7432e27fb0b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:06.690280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8eea90-b762-4163-9548-a7432e27fb0b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:06.690601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8eea90-b762-4163-9548-a7432e27fb0b None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:06.691275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8eea90-b762-4163-9548-a7432e27fb0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:06.692639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8eea90-b762-4163-9548-a7432e27fb0b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:06.699698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 467/944] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 09:36:06 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:36:07.470667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adbf17-170e-45c3-854f-057fea7c0e2d None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:07.471219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adbf17-170e-45c3-854f-057fea7c0e2d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:07.471219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adbf17-170e-45c3-854f-057fea7c0e2d None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:07.472924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 478/945] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:07.478466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bfc24-8bb8-4e44-88de-a6604ceb0a9f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:07.478975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bfc24-8bb8-4e44-88de-a6604ceb0a9f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:07.478975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bfc24-8bb8-4e44-88de-a6604ceb0a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:07.498405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b39bfc24-8bb8-4e44-88de-a6604ceb0a9f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:07.499619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b39bfc24-8bb8-4e44-88de-a6604ceb0a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:07.576274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 468/946] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:07.739179 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72722f4a-8bb3-41ac-94f8-9d54cc3482e3 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:07.739375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72722f4a-8bb3-41ac-94f8-9d54cc3482e3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:07.739669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72722f4a-8bb3-41ac-94f8-9d54cc3482e3 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:07.741227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 479/947] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:07.747097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1564a9-7156-4f27-915c-e088a0ef49d2 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:07.747653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1564a9-7156-4f27-915c-e088a0ef49d2 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:07.748152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1564a9-7156-4f27-915c-e088a0ef49d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:07.767554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ae1564a9-7156-4f27-915c-e088a0ef49d2 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:07.769139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ae1564a9-7156-4f27-915c-e088a0ef49d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:07.799824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:07.843827 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 469/948] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:07.861049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72722f4a-8bb3-41ac-94f8-9d54cc3482e3 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:07.967751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72722f4a-8bb3-41ac-94f8-9d54cc3482e3 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:07.994335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:08.005710 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba8e43-f706-4e0e-96a6-6a6873421273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:08.006355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba8e43-f706-4e0e-96a6-6a6873421273 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.006799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba8e43-f706-4e0e-96a6-6a6873421273 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.007086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba8e43-f706-4e0e-96a6-6a6873421273 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.019278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba8e43-f706-4e0e-96a6-6a6873421273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-1fba8e43-f706-4e0e-96a6-6a6873421273 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:08.024017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:08.025938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:08.027168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 480/949] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Aug 31 09:36:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:08.034245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1564a9-7156-4f27-915c-e088a0ef49d2 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:08.114517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba8e43-f706-4e0e-96a6-6a6873421273 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:08.161787 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1564a9-7156-4f27-915c-e088a0ef49d2 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:08.202628 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dd302-9d5e-455b-a696-50d801d21764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:08.203275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dd302-9d5e-455b-a696-50d801d21764 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.203552 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dd302-9d5e-455b-a696-50d801d21764 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.203824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dd302-9d5e-455b-a696-50d801d21764 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.204498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dd302-9d5e-455b-a696-50d801d21764 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:08.204791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dd302-9d5e-455b-a696-50d801d21764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:08.206165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dd302-9d5e-455b-a696-50d801d21764 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:08.218653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 470/950] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 09:36:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:08.229269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dd302-9d5e-455b-a696-50d801d21764 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:08.243580 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba8e43-f706-4e0e-96a6-6a6873421273 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:08.273048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:08.287819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4861b-31d3-4918-b4c9-b6adc42c70b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:08.288435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4861b-31d3-4918-b4c9-b6adc42c70b4 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.288718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4861b-31d3-4918-b4c9-b6adc42c70b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.289060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4861b-31d3-4918-b4c9-b6adc42c70b4 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.290448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4861b-31d3-4918-b4c9-b6adc42c70b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:08.292027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4861b-31d3-4918-b4c9-b6adc42c70b4 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:08.294990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: WARNING keystone.server.flask.common [None req-1bd4861b-31d3-4918-b4c9-b6adc42c70b4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 09:36:08.316938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.sql.core [None req-1bd4861b-31d3-4918-b4c9-b6adc42c70b4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 09:36:08.316938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 09:36:08.316938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [parameters: {'id': '7934fc1ec91a4770bd1327456ef04d2d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 09:36:08.316938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93532) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-1bd4861b-31d3-4918-b4c9-b6adc42c70b4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7934fc1ec91a4770bd1327456ef04d2d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7934fc1ec91a4770bd1327456ef04d2d. Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:36:08.340969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application result.read() Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 09:36:08.341852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application yield resource Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise self.value Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application session.commit() Aug 31 09:36:08.342789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self.session.flush() Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application compat.raise_( Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 09:36:08.344290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 09:36:08.345165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application result.read() Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:36:08.345939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [parameters: {'id': '7934fc1ec91a4770bd1327456ef04d2d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:08.347802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:08.349781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:08.351922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:08.351922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 09:36:08.351922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 09:36:08.351922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:08.351922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:08.351922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 09:36:08.351922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 09:36:08.351922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7934fc1ec91a4770bd1327456ef04d2d. Aug 31 09:36:08.351922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:08.351922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 481/951] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Aug 31 09:36:08 2022] POST /identity/v3/projects => generated 262 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:08.351922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4861b-31d3-4918-b4c9-b6adc42c70b4 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:08.374025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dd302-9d5e-455b-a696-50d801d21764 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:08.408064 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:08.415372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cf66b-b556-471c-a08e-64b345e33cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:08.416730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cf66b-b556-471c-a08e-64b345e33cc2 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.417232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cf66b-b556-471c-a08e-64b345e33cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.417662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cf66b-b556-471c-a08e-64b345e33cc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.434191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cf66b-b556-471c-a08e-64b345e33cc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:08.436065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cf66b-b556-471c-a08e-64b345e33cc2 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-ff0cf66b-b556-471c-a08e-64b345e33cc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:08.452779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:08.453837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:08.454801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:08.454801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:08.454801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:08.462577 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 471/952] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:08.464658 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cf66b-b556-471c-a08e-64b345e33cc2 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:08.481616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4861b-31d3-4918-b4c9-b6adc42c70b4 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:08.530831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61074d53-c016-40aa-b80c-0911739bf8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:08.531425 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61074d53-c016-40aa-b80c-0911739bf8a5 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.531708 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61074d53-c016-40aa-b80c-0911739bf8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.532045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61074d53-c016-40aa-b80c-0911739bf8a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.539270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61074d53-c016-40aa-b80c-0911739bf8a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:08.541764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61074d53-c016-40aa-b80c-0911739bf8a5 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-61074d53-c016-40aa-b80c-0911739bf8a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:08.553563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:08.554588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:08.556155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:08.556155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:08.556155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:08.556155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:08.556155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:08.556155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:08.556155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:08.556155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:08.556155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 482/953] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:08.556644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d99c04-65ac-401f-bc14-eced07ee8d48 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.557211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d99c04-65ac-401f-bc14-eced07ee8d48 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.557319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d99c04-65ac-401f-bc14-eced07ee8d48 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.561396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 483/954] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:08.562154 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:08.562595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d99c04-65ac-401f-bc14-eced07ee8d48 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:08.586423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cf66b-b556-471c-a08e-64b345e33cc2 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:08.625690 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd2a7e-843f-4751-8c0e-f6a1e6067ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:08.626303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd2a7e-843f-4751-8c0e-f6a1e6067ed9 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.626578 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd2a7e-843f-4751-8c0e-f6a1e6067ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.626845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd2a7e-843f-4751-8c0e-f6a1e6067ed9 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.627510 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd2a7e-843f-4751-8c0e-f6a1e6067ed9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:08.627807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd2a7e-843f-4751-8c0e-f6a1e6067ed9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:08.629161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd2a7e-843f-4751-8c0e-f6a1e6067ed9 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:08.645811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 472/955] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:08.645811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bdbd7-31ec-43a7-baea-95ece30cdd1c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.645811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bdbd7-31ec-43a7-baea-95ece30cdd1c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.645811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bdbd7-31ec-43a7-baea-95ece30cdd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.664547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-9c7bdbd7-31ec-43a7-baea-95ece30cdd1c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:08.666622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9c7bdbd7-31ec-43a7-baea-95ece30cdd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:08.675177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:08.679505 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d99c04-65ac-401f-bc14-eced07ee8d48 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:08.714352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cdcb1-8f8c-4d8d-a533-ea53213b857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:08.714963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cdcb1-8f8c-4d8d-a533-ea53213b857d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.715291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cdcb1-8f8c-4d8d-a533-ea53213b857d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.715564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cdcb1-8f8c-4d8d-a533-ea53213b857d None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.716231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cdcb1-8f8c-4d8d-a533-ea53213b857d None admin] RBAC: Adding query filter params (name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:08.716531 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cdcb1-8f8c-4d8d-a533-ea53213b857d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:08.717998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cdcb1-8f8c-4d8d-a533-ea53213b857d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:08.734678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 473/956] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:08.743770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 484/957] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:36:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:08.744120 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bdbd7-31ec-43a7-baea-95ece30cdd1c None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:08.744222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc292f-b2e8-476f-8244-df986eb76a2f None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.744222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc292f-b2e8-476f-8244-df986eb76a2f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.744222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc292f-b2e8-476f-8244-df986eb76a2f None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.744222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 485/958] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:08.744735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4c5e3-0473-436a-b509-8b36bae7c8e9 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.747163 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4c5e3-0473-436a-b509-8b36bae7c8e9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.747163 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4c5e3-0473-436a-b509-8b36bae7c8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.766059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0eb4c5e3-0473-436a-b509-8b36bae7c8e9 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:08.767720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0eb4c5e3-0473-436a-b509-8b36bae7c8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:08.843719 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:08.847611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 486/959] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:08.878751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bdbd7-31ec-43a7-baea-95ece30cdd1c None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:08.921295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8e5f5-189c-4e71-94e4-73042c9489bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:08.922146 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8e5f5-189c-4e71-94e4-73042c9489bf None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.922435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8e5f5-189c-4e71-94e4-73042c9489bf None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.922806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8e5f5-189c-4e71-94e4-73042c9489bf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.933157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8e5f5-189c-4e71-94e4-73042c9489bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:08.935037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8e5f5-189c-4e71-94e4-73042c9489bf None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-bca8e5f5-189c-4e71-94e4-73042c9489bf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:36:08.949597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:08.950619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:08.951613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:08.951613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:08.951613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:36:08.951613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:36:08.951613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:36:08.951613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:36:08.951613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:36:08.951613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:36:08.951613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:36:08.951613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:08.951613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 474/960] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 09:36:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:08.957471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4c5e3-0473-436a-b509-8b36bae7c8e9 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:09.036554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8e5f5-189c-4e71-94e4-73042c9489bf None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:09.079171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:09.090182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4c5e3-0473-436a-b509-8b36bae7c8e9 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:09.129715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ffc91-10de-454b-bb46-bdcaabb67021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:09.130239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ffc91-10de-454b-bb46-bdcaabb67021 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:09.130518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ffc91-10de-454b-bb46-bdcaabb67021 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:09.130786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ffc91-10de-454b-bb46-bdcaabb67021 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:09.131591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ffc91-10de-454b-bb46-bdcaabb67021 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:09.131894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ffc91-10de-454b-bb46-bdcaabb67021 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:09.133433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ffc91-10de-454b-bb46-bdcaabb67021 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:09.152640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 487/961] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Aug 31 09:36:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:09.153448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ffc91-10de-454b-bb46-bdcaabb67021 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:09.158649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8e5f5-189c-4e71-94e4-73042c9489bf None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:09.193656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:09.202227 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1141c-0256-49c2-9644-c816ba5c3b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:09.203027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1141c-0256-49c2-9644-c816ba5c3b49 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:09.203389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1141c-0256-49c2-9644-c816ba5c3b49 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:09.203657 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1141c-0256-49c2-9644-c816ba5c3b49 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:09.204300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d1141c-0256-49c2-9644-c816ba5c3b49 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:09.206436 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d1141c-0256-49c2-9644-c816ba5c3b49 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-35d1141c-0256-49c2-9644-c816ba5c3b49 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:09.217303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:09.219369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:09.220297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 475/962] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:09.221125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1141c-0256-49c2-9644-c816ba5c3b49 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:09.280385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ffc91-10de-454b-bb46-bdcaabb67021 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:09.320587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb7475-0c71-4648-bf29-7da1f831f4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:09.321794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb7475-0c71-4648-bf29-7da1f831f4de None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:09.322074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb7475-0c71-4648-bf29-7da1f831f4de None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:09.322351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb7475-0c71-4648-bf29-7da1f831f4de None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:09.322947 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb7475-0c71-4648-bf29-7da1f831f4de None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:09.324488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb7475-0c71-4648-bf29-7da1f831f4de None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-91cb7475-0c71-4648-bf29-7da1f831f4de None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:09.333974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:09.341023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:09.342112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 488/963] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 09:36:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:09.343186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb7475-0c71-4648-bf29-7da1f831f4de None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:09.352312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:09.354010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1141c-0256-49c2-9644-c816ba5c3b49 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:09.399299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab1a10-a18e-4cde-b590-795b8c9555ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:09.399299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab1a10-a18e-4cde-b590-795b8c9555ca None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:09.399299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab1a10-a18e-4cde-b590-795b8c9555ca None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:09.400265 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab1a10-a18e-4cde-b590-795b8c9555ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:09.466025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb7475-0c71-4648-bf29-7da1f831f4de None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:09.470635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:09.478271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab1a10-a18e-4cde-b590-795b8c9555ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:09.480332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab1a10-a18e-4cde-b590-795b8c9555ca None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:09.507701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89beb26-a431-45ed-917b-cd7cb1fd8d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:09.508432 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89beb26-a431-45ed-917b-cd7cb1fd8d6a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:09.509316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89beb26-a431-45ed-917b-cd7cb1fd8d6a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:09.509743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89beb26-a431-45ed-917b-cd7cb1fd8d6a None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:09.510387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89beb26-a431-45ed-917b-cd7cb1fd8d6a None admin] RBAC: Adding query filter params (name=compute) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:09.510699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89beb26-a431-45ed-917b-cd7cb1fd8d6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:09.512108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89beb26-a431-45ed-917b-cd7cb1fd8d6a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:09.517658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 489/964] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:36:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:09.518637 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89beb26-a431-45ed-917b-cd7cb1fd8d6a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:09.596840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 476/965] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:09 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:36:09.596840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab1a10-a18e-4cde-b590-795b8c9555ca None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:09.610484 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:09.641283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89beb26-a431-45ed-917b-cd7cb1fd8d6a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:09.678345 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4870f7c-4617-43e6-b5a0-e21feaf5932f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:09.678957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4870f7c-4617-43e6-b5a0-e21feaf5932f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:09.679487 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4870f7c-4617-43e6-b5a0-e21feaf5932f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:09.679597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4870f7c-4617-43e6-b5a0-e21feaf5932f None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:09.680252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4870f7c-4617-43e6-b5a0-e21feaf5932f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:09.681202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4870f7c-4617-43e6-b5a0-e21feaf5932f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:09.682648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4870f7c-4617-43e6-b5a0-e21feaf5932f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:09.687765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 490/966] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 09:36:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:09.688483 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4870f7c-4617-43e6-b5a0-e21feaf5932f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:09.698097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:09.705214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab1a10-a18e-4cde-b590-795b8c9555ca None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:09.741071 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac264e-a00f-4d73-88da-6a78fa9930b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:09.741674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac264e-a00f-4d73-88da-6a78fa9930b5 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:09.741951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac264e-a00f-4d73-88da-6a78fa9930b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:09.742478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac264e-a00f-4d73-88da-6a78fa9930b5 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:09.742905 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac264e-a00f-4d73-88da-6a78fa9930b5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:09.743299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac264e-a00f-4d73-88da-6a78fa9930b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:09.744789 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac264e-a00f-4d73-88da-6a78fa9930b5 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:09.749736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 477/967] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:36:09 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:09.750470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac264e-a00f-4d73-88da-6a78fa9930b5 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:09.839636 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4870f7c-4617-43e6-b5a0-e21feaf5932f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:09.876321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac264e-a00f-4d73-88da-6a78fa9930b5 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:09.891590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeeb62b-daad-4d16-9b37-3833d09f1dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:09.892790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeb62b-daad-4d16-9b37-3833d09f1dfe None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:09.893780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeb62b-daad-4d16-9b37-3833d09f1dfe None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:09.894248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeb62b-daad-4d16-9b37-3833d09f1dfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:09.908819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:09.911958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeeb62b-daad-4d16-9b37-3833d09f1dfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:09.913417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeeb62b-daad-4d16-9b37-3833d09f1dfe None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:09.921623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70871f3-6cc4-485b-af2d-01af5a5935eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:09.922236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70871f3-6cc4-485b-af2d-01af5a5935eb None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:09.922618 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70871f3-6cc4-485b-af2d-01af5a5935eb None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:09.922895 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70871f3-6cc4-485b-af2d-01af5a5935eb None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:09.923556 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70871f3-6cc4-485b-af2d-01af5a5935eb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:09.923864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70871f3-6cc4-485b-af2d-01af5a5935eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:09.925351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70871f3-6cc4-485b-af2d-01af5a5935eb None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-6eeeb62b-daad-4d16-9b37-3833d09f1dfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:09.931088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:09.932055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:09.932946 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:09.932946 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:09.932946 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:09.933114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 478/968] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 09:36:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:09.933193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 491/969] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:09.938777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70871f3-6cc4-485b-af2d-01af5a5935eb None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:09.938930 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeeb62b-daad-4d16-9b37-3833d09f1dfe None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:09.971490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:10.051592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70871f3-6cc4-485b-af2d-01af5a5935eb None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:10.066889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeeb62b-daad-4d16-9b37-3833d09f1dfe None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:10.093205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aadeb5-3207-413a-b023-b14e57638623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:10.093917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aadeb5-3207-413a-b023-b14e57638623 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:10.094195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aadeb5-3207-413a-b023-b14e57638623 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:10.094463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aadeb5-3207-413a-b023-b14e57638623 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:10.095251 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aadeb5-3207-413a-b023-b14e57638623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:10.095553 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aadeb5-3207-413a-b023-b14e57638623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:10.096916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aadeb5-3207-413a-b023-b14e57638623 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:10.107017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c798a88-7669-4ea6-80c9-edcde9a90d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:10.107787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c798a88-7669-4ea6-80c9-edcde9a90d0e None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:10.108162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c798a88-7669-4ea6-80c9-edcde9a90d0e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:10.108550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c798a88-7669-4ea6-80c9-edcde9a90d0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:10.109325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c798a88-7669-4ea6-80c9-edcde9a90d0e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:10.110206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 479/970] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:10.110889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c798a88-7669-4ea6-80c9-edcde9a90d0e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:10.115807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aadeb5-3207-413a-b023-b14e57638623 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:10.135540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 492/971] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:36:09 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:10.239900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:10.245654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aadeb5-3207-413a-b023-b14e57638623 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:10.293426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17cc76-bf30-4600-8f56-70f48ca714d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:10.294063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17cc76-bf30-4600-8f56-70f48ca714d6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:10.294334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17cc76-bf30-4600-8f56-70f48ca714d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:10.294597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17cc76-bf30-4600-8f56-70f48ca714d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:10.302666 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17cc76-bf30-4600-8f56-70f48ca714d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:10.304259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17cc76-bf30-4600-8f56-70f48ca714d6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-cd17cc76-bf30-4600-8f56-70f48ca714d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:10.316096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:10.317094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:10.318073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:10.318073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:10.318073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:10.318073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:10.318073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:10.318073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:10.318073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:10.318073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:10.318073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 480/972] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:10.323879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c798a88-7669-4ea6-80c9-edcde9a90d0e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:10.379601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:10.435432 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c798a88-7669-4ea6-80c9-edcde9a90d0e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:10.471447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4672389-d92d-4307-a07e-3e216a758f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:10.472065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4672389-d92d-4307-a07e-3e216a758f3c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:10.472342 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4672389-d92d-4307-a07e-3e216a758f3c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:10.472615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4672389-d92d-4307-a07e-3e216a758f3c None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:10.473310 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4672389-d92d-4307-a07e-3e216a758f3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:10.473616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4672389-d92d-4307-a07e-3e216a758f3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:10.475053 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4672389-d92d-4307-a07e-3e216a758f3c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:10.488478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 493/973] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:36:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:10.494938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17cc76-bf30-4600-8f56-70f48ca714d6 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:10.601346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17cc76-bf30-4600-8f56-70f48ca714d6 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:10.649680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29d51f-901e-44e4-9bb8-eb97286f757e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:10.649680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29d51f-901e-44e4-9bb8-eb97286f757e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:10.649680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29d51f-901e-44e4-9bb8-eb97286f757e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:10.649680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29d51f-901e-44e4-9bb8-eb97286f757e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:10.650712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29d51f-901e-44e4-9bb8-eb97286f757e None admin] RBAC: Adding query filter params (role.id=5bb2daf5cef542028e8802f2f06a46bd, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=0a2e5c1014f1441aa47d5de46462b6d9) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:10.650712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29d51f-901e-44e4-9bb8-eb97286f757e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:10.650712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29d51f-901e-44e4-9bb8-eb97286f757e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:10.656703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 481/974] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:10 2022] GET /identity/v3/role_assignments?user.id=0a2e5c1014f1441aa47d5de46462b6d9&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=5bb2daf5cef542028e8802f2f06a46bd => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:10.857998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de452f8b-3e49-4956-b46a-c010b831b07f None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:10.858501 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de452f8b-3e49-4956-b46a-c010b831b07f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:10.858501 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de452f8b-3e49-4956-b46a-c010b831b07f None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:10.860203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 494/975] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:10.865703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeed81d-4625-4435-a57d-69492489583f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:10.865983 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeed81d-4625-4435-a57d-69492489583f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:10.866267 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeed81d-4625-4435-a57d-69492489583f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:10.873864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:10.886660 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-deeed81d-4625-4435-a57d-69492489583f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:10.887919 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-deeed81d-4625-4435-a57d-69492489583f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:10.966190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 482/976] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:11.248502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de452f8b-3e49-4956-b46a-c010b831b07f None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:11.364389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de452f8b-3e49-4956-b46a-c010b831b07f None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:11.375056 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:11.401586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771a9dc-fd1c-4170-bf01-2643daae3147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:11.402200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771a9dc-fd1c-4170-bf01-2643daae3147 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:11.402487 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771a9dc-fd1c-4170-bf01-2643daae3147 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:11.402761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771a9dc-fd1c-4170-bf01-2643daae3147 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:11.403441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771a9dc-fd1c-4170-bf01-2643daae3147 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:11.404871 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771a9dc-fd1c-4170-bf01-2643daae3147 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:11.421706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.sql.core [None req-f771a9dc-fd1c-4170-bf01-2643daae3147 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 09:36:11.421706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 09:36:11.421706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [parameters: {'user_id': '5fd26272b9db4d9e99be3ae6f6ebcbf4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 09:36:11.421706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93532) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-f771a9dc-fd1c-4170-bf01-2643daae3147 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application result.read() Aug 31 09:36:11.428473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:36:11.429375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application yield resource Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise self.value Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application session.commit() Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 09:36:11.430585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self.session.flush() Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application compat.raise_( Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 09:36:11.433713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:36:11.435499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application result.read() Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:36:11.436647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [parameters: {'user_id': '5fd26272b9db4d9e99be3ae6f6ebcbf4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:11.437701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 09:36:11.438974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 09:36:11.441129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 09:36:11.441129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 09:36:11.441129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 09:36:11.441129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 09:36:11.441129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 09:36:11.441129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:11.441129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 495/977] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Aug 31 09:36:11 2022] POST /identity/v3/users => generated 160 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:11.442036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeed81d-4625-4435-a57d-69492489583f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:11.547185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeed81d-4625-4435-a57d-69492489583f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:11.583486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96855a-1167-49ea-b6f2-1916bb5e57dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:11.584115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96855a-1167-49ea-b6f2-1916bb5e57dc None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:11.584388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96855a-1167-49ea-b6f2-1916bb5e57dc None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:11.584647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96855a-1167-49ea-b6f2-1916bb5e57dc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:11.586651 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:11.592901 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96855a-1167-49ea-b6f2-1916bb5e57dc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:11.594358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96855a-1167-49ea-b6f2-1916bb5e57dc None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-fc96855a-1167-49ea-b6f2-1916bb5e57dc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:36:11.608364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:11.611078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:11.613280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:11.613280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:11.613280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:36:11.613280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:36:11.613280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:36:11.613280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:36:11.613280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:36:11.613280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:36:11.613280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:36:11.613280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:11.613280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 483/978] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:11.619148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771a9dc-fd1c-4170-bf01-2643daae3147 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:11.727053 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771a9dc-fd1c-4170-bf01-2643daae3147 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:11.772395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be7661-7abf-4f81-b8b1-5207bddf74ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:11.773358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be7661-7abf-4f81-b8b1-5207bddf74ce None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:11.773795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be7661-7abf-4f81-b8b1-5207bddf74ce None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:11.774243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be7661-7abf-4f81-b8b1-5207bddf74ce None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:11.775837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be7661-7abf-4f81-b8b1-5207bddf74ce None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:11.776324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be7661-7abf-4f81-b8b1-5207bddf74ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:11.778153 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be7661-7abf-4f81-b8b1-5207bddf74ce None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:11.822463 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 496/979] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 09:36:11 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:11.830358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c60ca7-d3e7-4639-b773-f32a962cc6e0 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:11.830791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c60ca7-d3e7-4639-b773-f32a962cc6e0 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:11.831190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c60ca7-d3e7-4639-b773-f32a962cc6e0 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:11.833136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 484/980] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:11.839806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5447086-f6fd-4f1f-bce5-e833f0f3f830 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:11.840405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5447086-f6fd-4f1f-bce5-e833f0f3f830 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:11.840849 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5447086-f6fd-4f1f-bce5-e833f0f3f830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:11.864062 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-d5447086-f6fd-4f1f-bce5-e833f0f3f830 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:11.866377 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d5447086-f6fd-4f1f-bce5-e833f0f3f830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:11.896853 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:11.953894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 497/981] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:12.227043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c60ca7-d3e7-4639-b773-f32a962cc6e0 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:12.296174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3feec57-f670-4b0a-9b96-4a56f3334299 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:12.296475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3feec57-f670-4b0a-9b96-4a56f3334299 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:12.296751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3feec57-f670-4b0a-9b96-4a56f3334299 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:12.298254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 498/982] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:12.303728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68be073-796a-41d8-8295-bb5ad8bafb36 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:12.304015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68be073-796a-41d8-8295-bb5ad8bafb36 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:12.304292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68be073-796a-41d8-8295-bb5ad8bafb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:12.324774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-d68be073-796a-41d8-8295-bb5ad8bafb36 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:12.325994 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d68be073-796a-41d8-8295-bb5ad8bafb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:12.341090 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c60ca7-d3e7-4639-b773-f32a962cc6e0 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:12.378509 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4f6c9-341c-4aaf-9fab-b0eb343a2b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:12.379156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4f6c9-341c-4aaf-9fab-b0eb343a2b4c None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:12.379427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4f6c9-341c-4aaf-9fab-b0eb343a2b4c None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:12.379697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4f6c9-341c-4aaf-9fab-b0eb343a2b4c None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:12.380378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4f6c9-341c-4aaf-9fab-b0eb343a2b4c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:12.381763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4f6c9-341c-4aaf-9fab-b0eb343a2b4c None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:12.399769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 485/983] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:36:12 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:12.420493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 499/984] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:12.694744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c04186-3499-4717-8160-c372924f8d56 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:12.695337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c04186-3499-4717-8160-c372924f8d56 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:12.695337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c04186-3499-4717-8160-c372924f8d56 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:12.697462 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 486/985] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:12.697714 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68be073-796a-41d8-8295-bb5ad8bafb36 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:12.702491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b730c0-d1bc-4f4b-abb6-75e8d1348c8d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:12.702771 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b730c0-d1bc-4f4b-abb6-75e8d1348c8d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:12.703042 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b730c0-d1bc-4f4b-abb6-75e8d1348c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:12.722991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-d9b730c0-d1bc-4f4b-abb6-75e8d1348c8d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:12.724221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d9b730c0-d1bc-4f4b-abb6-75e8d1348c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:12.786252 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:12.803332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:12.810109 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 487/986] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:12.828690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68be073-796a-41d8-8295-bb5ad8bafb36 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:12.869412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce230fdd-2b1a-4d45-884d-e570a76f3988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:12.870096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce230fdd-2b1a-4d45-884d-e570a76f3988 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:12.870330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce230fdd-2b1a-4d45-884d-e570a76f3988 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:12.870593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce230fdd-2b1a-4d45-884d-e570a76f3988 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:12.871219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce230fdd-2b1a-4d45-884d-e570a76f3988 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:12.872602 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce230fdd-2b1a-4d45-884d-e570a76f3988 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-ce230fdd-2b1a-4d45-884d-e570a76f3988 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:12.882318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:12.883770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:36:12.884782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:12.887493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 500/987] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 09:36:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:12.890235 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b730c0-d1bc-4f4b-abb6-75e8d1348c8d None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:12.998493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b730c0-d1bc-4f4b-abb6-75e8d1348c8d None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:13.033844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4b348-a844-45e9-b4cd-23acafb8d2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:13.034453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4b348-a844-45e9-b4cd-23acafb8d2a3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:13.034733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4b348-a844-45e9-b4cd-23acafb8d2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:13.035003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4b348-a844-45e9-b4cd-23acafb8d2a3 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:13.035662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4b348-a844-45e9-b4cd-23acafb8d2a3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:13.035961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4b348-a844-45e9-b4cd-23acafb8d2a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:13.037333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4b348-a844-45e9-b4cd-23acafb8d2a3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:13.047427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 488/988] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 09:36:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:13.048988 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce230fdd-2b1a-4d45-884d-e570a76f3988 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:13.084006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4b348-a844-45e9-b4cd-23acafb8d2a3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:13.136889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:13.177004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce230fdd-2b1a-4d45-884d-e570a76f3988 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:13.190410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4b348-a844-45e9-b4cd-23acafb8d2a3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:13.215593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83017b2d-4225-474d-924d-e316e2401885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:13.216320 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83017b2d-4225-474d-924d-e316e2401885 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:13.216607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83017b2d-4225-474d-924d-e316e2401885 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:13.216941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83017b2d-4225-474d-924d-e316e2401885 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:13.217515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83017b2d-4225-474d-924d-e316e2401885 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:13.217822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83017b2d-4225-474d-924d-e316e2401885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:13.219288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83017b2d-4225-474d-924d-e316e2401885 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:13.230054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 501/989] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 09:36:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:13.230268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe20dbf-aed4-4a50-937f-a71fe2f61f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:13.230268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe20dbf-aed4-4a50-937f-a71fe2f61f13 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:13.230268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe20dbf-aed4-4a50-937f-a71fe2f61f13 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:13.230268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe20dbf-aed4-4a50-937f-a71fe2f61f13 None admin] PATH_INFO: `/v3/services/image` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:13.230649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe20dbf-aed4-4a50-937f-a71fe2f61f13 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:13.232079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe20dbf-aed4-4a50-937f-a71fe2f61f13 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:13.232524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83017b2d-4225-474d-924d-e316e2401885 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-dbe20dbf-aed4-4a50-937f-a71fe2f61f13 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:13.241385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:13.242428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:13.244003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 489/990] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Aug 31 09:36:13 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:13.249099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe20dbf-aed4-4a50-937f-a71fe2f61f13 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:13.250616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:13.345555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83017b2d-4225-474d-924d-e316e2401885 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:13.356406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe20dbf-aed4-4a50-937f-a71fe2f61f13 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:13.369709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:13.386793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c507a1-694b-400b-8f72-d7b6fba34570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:13.387638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c507a1-694b-400b-8f72-d7b6fba34570 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:13.388041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c507a1-694b-400b-8f72-d7b6fba34570 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:13.388313 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c507a1-694b-400b-8f72-d7b6fba34570 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:13.389055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c507a1-694b-400b-8f72-d7b6fba34570 None admin] RBAC: Adding query filter params (interface=admin, service_id=978f8ee4897346f2b744454d818ae195, region_id=RegionOne) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:13.389604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c507a1-694b-400b-8f72-d7b6fba34570 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:13.390970 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c507a1-694b-400b-8f72-d7b6fba34570 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:13.393718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e002166-c83c-4624-be42-f35d5f446841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:13.394538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e002166-c83c-4624-be42-f35d5f446841 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:13.395016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e002166-c83c-4624-be42-f35d5f446841 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:13.395477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e002166-c83c-4624-be42-f35d5f446841 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:13.396303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e002166-c83c-4624-be42-f35d5f446841 None admin] RBAC: Adding query filter params (name=image) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:13.396757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 502/991] 10.222.0.36 () {56 vars in 1249 bytes} [Wed Aug 31 09:36:13 2022] GET /identity/v3/endpoints?service_id=978f8ee4897346f2b744454d818ae195&interface=admin®ion_id=RegionOne => generated 193 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:13.396931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e002166-c83c-4624-be42-f35d5f446841 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:13.398315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e002166-c83c-4624-be42-f35d5f446841 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:13.402591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c507a1-694b-400b-8f72-d7b6fba34570 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:13.406057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 490/992] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:13.411251 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e002166-c83c-4624-be42-f35d5f446841 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:13.490287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:13.518369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c507a1-694b-400b-8f72-d7b6fba34570 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:13.528612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e002166-c83c-4624-be42-f35d5f446841 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:13.558118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fbf47-9b6a-44bc-80e9-0988d311458d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:13.558735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fbf47-9b6a-44bc-80e9-0988d311458d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:13.559018 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fbf47-9b6a-44bc-80e9-0988d311458d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:13.559329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fbf47-9b6a-44bc-80e9-0988d311458d None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:13.559936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fbf47-9b6a-44bc-80e9-0988d311458d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:13.561308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fbf47-9b6a-44bc-80e9-0988d311458d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:13.567740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 503/993] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 09:36:13 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:36:13.568644 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd46d9d-aa83-4205-9fda-6d60e40f0ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:13.569447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd46d9d-aa83-4205-9fda-6d60e40f0ea7 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:13.569922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd46d9d-aa83-4205-9fda-6d60e40f0ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:13.570465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd46d9d-aa83-4205-9fda-6d60e40f0ea7 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:13.571483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd46d9d-aa83-4205-9fda-6d60e40f0ea7 None admin] RBAC: Adding query filter params (type=image) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:13.571979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd46d9d-aa83-4205-9fda-6d60e40f0ea7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:13.573681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd46d9d-aa83-4205-9fda-6d60e40f0ea7 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:13.580192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 491/994] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:14.375766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98702376-a9c2-4b74-b54f-0edeb7c337ec None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:14.376281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98702376-a9c2-4b74-b54f-0edeb7c337ec None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:14.376281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98702376-a9c2-4b74-b54f-0edeb7c337ec None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:14.377747 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 504/995] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:14.382745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67a1f7-e375-4e10-8ab7-5cb420a7a3d9 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:14.383218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67a1f7-e375-4e10-8ab7-5cb420a7a3d9 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:14.383330 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67a1f7-e375-4e10-8ab7-5cb420a7a3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:14.402835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-cd67a1f7-e375-4e10-8ab7-5cb420a7a3d9 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:14.404077 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-cd67a1f7-e375-4e10-8ab7-5cb420a7a3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:14.441505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:14.477179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 492/996] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:14.747626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98702376-a9c2-4b74-b54f-0edeb7c337ec None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:14.861190 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98702376-a9c2-4b74-b54f-0edeb7c337ec None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:14.903704 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb0972-58f7-4437-bdb1-a36452559f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:14.904346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb0972-58f7-4437-bdb1-a36452559f6e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:14.904619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb0972-58f7-4437-bdb1-a36452559f6e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:14.904889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb0972-58f7-4437-bdb1-a36452559f6e None admin] PATH_INFO: `/v3/services/compute` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:14.905479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb0972-58f7-4437-bdb1-a36452559f6e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:14.906835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb0972-58f7-4437-bdb1-a36452559f6e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-30eb0972-58f7-4437-bdb1-a36452559f6e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:14.916687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:14.917730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:36:14.919807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:14.920923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 505/997] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:14.928297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67a1f7-e375-4e10-8ab7-5cb420a7a3d9 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:15.038825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67a1f7-e375-4e10-8ab7-5cb420a7a3d9 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:15.084749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87371c2e-3fbb-4383-8d6a-bd63691807ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:15.085579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87371c2e-3fbb-4383-8d6a-bd63691807ab None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:15.086057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87371c2e-3fbb-4383-8d6a-bd63691807ab None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:15.086548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87371c2e-3fbb-4383-8d6a-bd63691807ab None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:15.087574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87371c2e-3fbb-4383-8d6a-bd63691807ab None admin] RBAC: Adding query filter params (name=compute) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:15.088088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87371c2e-3fbb-4383-8d6a-bd63691807ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:15.089658 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87371c2e-3fbb-4383-8d6a-bd63691807ab None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:15.094765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 493/998] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:36:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:15.100795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb0972-58f7-4437-bdb1-a36452559f6e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:15.106325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:15.215933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb0972-58f7-4437-bdb1-a36452559f6e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:15.253781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09ccc2-04d7-4a97-ad91-87c2b33fbff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:15.254403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09ccc2-04d7-4a97-ad91-87c2b33fbff0 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:15.254677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09ccc2-04d7-4a97-ad91-87c2b33fbff0 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:15.254946 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09ccc2-04d7-4a97-ad91-87c2b33fbff0 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:15.255615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f09ccc2-04d7-4a97-ad91-87c2b33fbff0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:15.255913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f09ccc2-04d7-4a97-ad91-87c2b33fbff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:15.257283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f09ccc2-04d7-4a97-ad91-87c2b33fbff0 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:15.267739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 506/999] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:36:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:15.275385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87371c2e-3fbb-4383-8d6a-bd63691807ab None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:15.311421 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:15.385005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87371c2e-3fbb-4383-8d6a-bd63691807ab None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:15.422230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b59812-85a7-43e6-a5ed-bf50565f5ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:15.422839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b59812-85a7-43e6-a5ed-bf50565f5ad3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:15.423114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b59812-85a7-43e6-a5ed-bf50565f5ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:15.423410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b59812-85a7-43e6-a5ed-bf50565f5ad3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:15.424113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b59812-85a7-43e6-a5ed-bf50565f5ad3 None admin] RBAC: Adding query filter params (interface=public, service_id=3f01be9af864435a83c4b7ef58167448, region_id=RegionOne) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:15.424415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b59812-85a7-43e6-a5ed-bf50565f5ad3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:15.425781 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b59812-85a7-43e6-a5ed-bf50565f5ad3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:15.431388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 494/1000] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 09:36:15 2022] GET /identity/v3/endpoints?service_id=3f01be9af864435a83c4b7ef58167448&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:15.436982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09ccc2-04d7-4a97-ad91-87c2b33fbff0 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:15.509470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:15.520094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179817e-93b4-45cb-86b6-961d25e04c98 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:15.520314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179817e-93b4-45cb-86b6-961d25e04c98 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:15.520546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179817e-93b4-45cb-86b6-961d25e04c98 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:15.521884 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 495/1001] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:15.522927 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00392bd-0757-485a-99ff-f8cbc6ffa97a None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:15.523272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00392bd-0757-485a-99ff-f8cbc6ffa97a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:15.523549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00392bd-0757-485a-99ff-f8cbc6ffa97a None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:15.525328 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 496/1002] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:15.528096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ece160-5581-4c79-bed7-ee71f75242c3 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:15.528375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ece160-5581-4c79-bed7-ee71f75242c3 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:15.528641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ece160-5581-4c79-bed7-ee71f75242c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:15.547502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-e2ece160-5581-4c79-bed7-ee71f75242c3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:15.548712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e2ece160-5581-4c79-bed7-ee71f75242c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:15.555620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09ccc2-04d7-4a97-ad91-87c2b33fbff0 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:15.590476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d943b3-e991-4514-93bc-1dec6da8c6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:15.591000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d943b3-e991-4514-93bc-1dec6da8c6ee None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:15.591315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d943b3-e991-4514-93bc-1dec6da8c6ee None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:15.591589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d943b3-e991-4514-93bc-1dec6da8c6ee None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:15.592200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d943b3-e991-4514-93bc-1dec6da8c6ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:15.593569 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d943b3-e991-4514-93bc-1dec6da8c6ee None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:15.600062 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 507/1003] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 09:36:15 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:36:15.602565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d3584-f29c-47e8-af25-f5057b7cf937 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:15.602565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d3584-f29c-47e8-af25-f5057b7cf937 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:15.602565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d3584-f29c-47e8-af25-f5057b7cf937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:15.628000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 497/1004] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:15.640937 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-2f0d3584-f29c-47e8-af25-f5057b7cf937 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:15.642173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2f0d3584-f29c-47e8-af25-f5057b7cf937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:15.725209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 508/1005] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:15.901594 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ece160-5581-4c79-bed7-ee71f75242c3 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:15.929704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:15.991493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d3584-f29c-47e8-af25-f5057b7cf937 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:16.008189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ece160-5581-4c79-bed7-ee71f75242c3 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:16.046373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c0642-8f5c-4326-98fc-6e4724892f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:16.046989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c0642-8f5c-4326-98fc-6e4724892f1b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:16.047296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c0642-8f5c-4326-98fc-6e4724892f1b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:16.047561 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c0642-8f5c-4326-98fc-6e4724892f1b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:16.048142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c0642-8f5c-4326-98fc-6e4724892f1b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:16.049508 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c0642-8f5c-4326-98fc-6e4724892f1b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-9a6c0642-8f5c-4326-98fc-6e4724892f1b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:16.058483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:16.059779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:16.060964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 498/1006] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 09:36:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:16.065436 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c0642-8f5c-4326-98fc-6e4724892f1b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:16.075261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:16.125750 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d3584-f29c-47e8-af25-f5057b7cf937 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:16.166942 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee45bc-24cb-470d-8225-111ddd2b150d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:16.167528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee45bc-24cb-470d-8225-111ddd2b150d None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:16.167850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee45bc-24cb-470d-8225-111ddd2b150d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:16.168125 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee45bc-24cb-470d-8225-111ddd2b150d None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:16.168907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee45bc-24cb-470d-8225-111ddd2b150d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:16.170302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee45bc-24cb-470d-8225-111ddd2b150d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:16.173002 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:16.183984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c0642-8f5c-4326-98fc-6e4724892f1b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:16.198469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 509/1007] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:36:15 2022] POST /identity/v3/services => generated 242 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:36:16.232782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb17f8a-1b4e-4fe6-89d6-a2d3a2811f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:16.233482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb17f8a-1b4e-4fe6-89d6-a2d3a2811f7d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:16.233754 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb17f8a-1b4e-4fe6-89d6-a2d3a2811f7d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:16.234075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb17f8a-1b4e-4fe6-89d6-a2d3a2811f7d None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:16.234765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb17f8a-1b4e-4fe6-89d6-a2d3a2811f7d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:16.235213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb17f8a-1b4e-4fe6-89d6-a2d3a2811f7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:16.236697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb17f8a-1b4e-4fe6-89d6-a2d3a2811f7d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:16.242681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 499/1008] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 09:36:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:16.249304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee45bc-24cb-470d-8225-111ddd2b150d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:16.364125 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee45bc-24cb-470d-8225-111ddd2b150d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:16.388788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:16.403269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfcd4d1-0804-4cd2-9e6a-02d133304f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:16.403880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcd4d1-0804-4cd2-9e6a-02d133304f6d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:16.404155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcd4d1-0804-4cd2-9e6a-02d133304f6d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:16.404414 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcd4d1-0804-4cd2-9e6a-02d133304f6d None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:16.405049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfcd4d1-0804-4cd2-9e6a-02d133304f6d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:16.405346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfcd4d1-0804-4cd2-9e6a-02d133304f6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:16.406699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfcd4d1-0804-4cd2-9e6a-02d133304f6d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:16.412041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 510/1009] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 09:36:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:16.417646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb17f8a-1b4e-4fe6-89d6-a2d3a2811f7d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:16.520918 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb17f8a-1b4e-4fe6-89d6-a2d3a2811f7d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:16.556938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec02b0-def1-40ad-adfd-6d3f31522e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:16.557545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec02b0-def1-40ad-adfd-6d3f31522e6d None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:16.557816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec02b0-def1-40ad-adfd-6d3f31522e6d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:16.558082 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec02b0-def1-40ad-adfd-6d3f31522e6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:16.558757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec02b0-def1-40ad-adfd-6d3f31522e6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:16.560148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec02b0-def1-40ad-adfd-6d3f31522e6d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:16.578654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 500/1010] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:36:16 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:17.691610 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6488d-51f7-4ffd-9a94-b3302ade0029 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:17.692174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6488d-51f7-4ffd-9a94-b3302ade0029 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:17.692174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6488d-51f7-4ffd-9a94-b3302ade0029 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:17.693803 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 511/1011] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:17.698829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4394a8-5cc5-4077-8bef-f048aca34aec None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:17.699307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4394a8-5cc5-4077-8bef-f048aca34aec None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:17.699408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4394a8-5cc5-4077-8bef-f048aca34aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:17.719071 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-2b4394a8-5cc5-4077-8bef-f048aca34aec None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:17.720347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2b4394a8-5cc5-4077-8bef-f048aca34aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:17.730424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:17.800874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 501/1012] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:17 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:18.073200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6488d-51f7-4ffd-9a94-b3302ade0029 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:18.137175 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e6a7b-3ef7-4fa6-b35e-2f814de399b7 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.137456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e6a7b-3ef7-4fa6-b35e-2f814de399b7 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.137718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e6a7b-3ef7-4fa6-b35e-2f814de399b7 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.139053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 502/1013] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:18.144477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a27361-5ea4-4889-bb77-f5bb7502db0f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.144752 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a27361-5ea4-4889-bb77-f5bb7502db0f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.145023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a27361-5ea4-4889-bb77-f5bb7502db0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.166574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-17a27361-5ea4-4889-bb77-f5bb7502db0f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:18.167774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-17a27361-5ea4-4889-bb77-f5bb7502db0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:18.188559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6488d-51f7-4ffd-9a94-b3302ade0029 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:18.224314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da95669-0f75-4e7d-b65a-00ea481e178e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:18.225249 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da95669-0f75-4e7d-b65a-00ea481e178e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.225530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da95669-0f75-4e7d-b65a-00ea481e178e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.225805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da95669-0f75-4e7d-b65a-00ea481e178e None admin] PATH_INFO: `/v3/services/compute` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.226391 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da95669-0f75-4e7d-b65a-00ea481e178e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:18.227864 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da95669-0f75-4e7d-b65a-00ea481e178e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-1da95669-0f75-4e7d-b65a-00ea481e178e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:18.239561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:18.240606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:36:18.241467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:18.242222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 503/1014] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:18 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:18.247072 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 512/1015] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:18.252414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a27361-5ea4-4889-bb77-f5bb7502db0f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:18.325773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:18.360732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a27361-5ea4-4889-bb77-f5bb7502db0f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:18.395894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655493ca-43c7-4332-8958-b300573f9250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:18.396498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655493ca-43c7-4332-8958-b300573f9250 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.396770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655493ca-43c7-4332-8958-b300573f9250 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.397034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655493ca-43c7-4332-8958-b300573f9250 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.397656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655493ca-43c7-4332-8958-b300573f9250 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:18.397950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655493ca-43c7-4332-8958-b300573f9250 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:18.400133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655493ca-43c7-4332-8958-b300573f9250 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:18.405864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 504/1016] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:36:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:18.411318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da95669-0f75-4e7d-b65a-00ea481e178e None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:18.470962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:18.520640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655493ca-43c7-4332-8958-b300573f9250 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:18.545438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da95669-0f75-4e7d-b65a-00ea481e178e None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:18.584563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9922764-8f7a-4300-a10f-3034881dc504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:18.585104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9922764-8f7a-4300-a10f-3034881dc504 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.585378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9922764-8f7a-4300-a10f-3034881dc504 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.585650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9922764-8f7a-4300-a10f-3034881dc504 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.586315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9922764-8f7a-4300-a10f-3034881dc504 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:18.588459 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9922764-8f7a-4300-a10f-3034881dc504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:18.588610 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9922764-8f7a-4300-a10f-3034881dc504 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:18.600170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 513/1017] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 09:36:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:18.601052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9922764-8f7a-4300-a10f-3034881dc504 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:18.649787 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655493ca-43c7-4332-8958-b300573f9250 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:18.685382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:18.698041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceafcb2-1b86-43f9-a9e3-6166aa6fda22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:18.698693 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceafcb2-1b86-43f9-a9e3-6166aa6fda22 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.698970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceafcb2-1b86-43f9-a9e3-6166aa6fda22 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.699571 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceafcb2-1b86-43f9-a9e3-6166aa6fda22 None admin] PATH_INFO: `/v3/services/image` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.699854 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceafcb2-1b86-43f9-a9e3-6166aa6fda22 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:18.701240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceafcb2-1b86-43f9-a9e3-6166aa6fda22 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-fceafcb2-1b86-43f9-a9e3-6166aa6fda22 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:18.711305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:18.712952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:18.714546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 505/1018] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Aug 31 09:36:18 2022] GET /identity/v3/services/image => generated 86 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:18.717067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1eec1-ce25-4701-b764-4c8bd684a9dc None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.717067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1eec1-ce25-4701-b764-4c8bd684a9dc None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.717067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1eec1-ce25-4701-b764-4c8bd684a9dc None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.717067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 506/1019] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:18.720124 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1eec1-ce25-4701-b764-4c8bd684a9dc None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:18.724092 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9922764-8f7a-4300-a10f-3034881dc504 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:18.745182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:18.777045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c61ae-d62a-4420-a380-433cb23e1be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:18.778009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c61ae-d62a-4420-a380-433cb23e1be0 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.778449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c61ae-d62a-4420-a380-433cb23e1be0 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.778954 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c61ae-d62a-4420-a380-433cb23e1be0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.780224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c61ae-d62a-4420-a380-433cb23e1be0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:18.782851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c61ae-d62a-4420-a380-433cb23e1be0 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:18.808899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 514/1020] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:36:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:36:18.810460 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a2403-6d0d-4d9e-920a-71750a6280d4 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.810741 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a2403-6d0d-4d9e-920a-71750a6280d4 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.811016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a2403-6d0d-4d9e-920a-71750a6280d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.833531 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-072a2403-6d0d-4d9e-920a-71750a6280d4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:18.834973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-072a2403-6d0d-4d9e-920a-71750a6280d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:18.865406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1eec1-ce25-4701-b764-4c8bd684a9dc None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:18.912874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9a163-bc0a-479a-8f6c-b8204275f5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:18.913829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9a163-bc0a-479a-8f6c-b8204275f5b7 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.914110 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9a163-bc0a-479a-8f6c-b8204275f5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.914385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9a163-bc0a-479a-8f6c-b8204275f5b7 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.915100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9a163-bc0a-479a-8f6c-b8204275f5b7 None admin] RBAC: Adding query filter params (name=image) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:18.915992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9a163-bc0a-479a-8f6c-b8204275f5b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:18.917531 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9a163-bc0a-479a-8f6c-b8204275f5b7 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:18.927179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 507/1021] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:18.930680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9a163-bc0a-479a-8f6c-b8204275f5b7 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:18.942915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 515/1022] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:19.020154 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:19.085152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9a163-bc0a-479a-8f6c-b8204275f5b7 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:19.134387 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a1b09-5577-4703-baf2-815cb57964a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:19.135005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a1b09-5577-4703-baf2-815cb57964a4 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:19.135325 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a1b09-5577-4703-baf2-815cb57964a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:19.135588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a1b09-5577-4703-baf2-815cb57964a4 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:19.136226 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a1b09-5577-4703-baf2-815cb57964a4 None admin] RBAC: Adding query filter params (type=image) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:19.136521 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a1b09-5577-4703-baf2-815cb57964a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:19.137904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a1b09-5577-4703-baf2-815cb57964a4 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:19.145170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 508/1023] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:19.152721 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a2403-6d0d-4d9e-920a-71750a6280d4 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:19.252206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a1b09-5577-4703-baf2-815cb57964a4 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:19.265658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:19.269029 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a2403-6d0d-4d9e-920a-71750a6280d4 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:19.309177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51444f7-be1b-4cf9-a62d-46f59293c1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:19.309788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51444f7-be1b-4cf9-a62d-46f59293c1e6 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:19.310059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51444f7-be1b-4cf9-a62d-46f59293c1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:19.310323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51444f7-be1b-4cf9-a62d-46f59293c1e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:19.311026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51444f7-be1b-4cf9-a62d-46f59293c1e6 None admin] RBAC: Adding query filter params (interface=public, service_id=35279baed58d4a9685d35c568143d06c, region_id=RegionOne) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:19.311380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51444f7-be1b-4cf9-a62d-46f59293c1e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:19.312896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51444f7-be1b-4cf9-a62d-46f59293c1e6 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:19.320490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 516/1024] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 09:36:19 2022] GET /identity/v3/endpoints?service_id=35279baed58d4a9685d35c568143d06c&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:19.327490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51444f7-be1b-4cf9-a62d-46f59293c1e6 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:19.385249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a1b09-5577-4703-baf2-815cb57964a4 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:19.410877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:19.427742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7f96e-ec44-49db-a7d8-60b039acb744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:19.428530 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7f96e-ec44-49db-a7d8-60b039acb744 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:19.428814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7f96e-ec44-49db-a7d8-60b039acb744 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:19.429089 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7f96e-ec44-49db-a7d8-60b039acb744 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:19.430847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7f96e-ec44-49db-a7d8-60b039acb744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:19.432305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7f96e-ec44-49db-a7d8-60b039acb744 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:19.433979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 509/1025] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:19.439880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7f96e-ec44-49db-a7d8-60b039acb744 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:19.443624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51444f7-be1b-4cf9-a62d-46f59293c1e6 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:19.485922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cda45-2ee1-42f3-8848-67f6228d6868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:19.486963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cda45-2ee1-42f3-8848-67f6228d6868 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:19.487452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cda45-2ee1-42f3-8848-67f6228d6868 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:19.487913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cda45-2ee1-42f3-8848-67f6228d6868 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:19.488538 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cda45-2ee1-42f3-8848-67f6228d6868 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:19.489921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cda45-2ee1-42f3-8848-67f6228d6868 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:19.497825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 517/1026] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 09:36:19 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:36:19.573648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7f96e-ec44-49db-a7d8-60b039acb744 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:19.615202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa6696-1075-46bc-8889-b3ac08821377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:19.616192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa6696-1075-46bc-8889-b3ac08821377 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:19.616662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa6696-1075-46bc-8889-b3ac08821377 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:19.617117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa6696-1075-46bc-8889-b3ac08821377 None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:19.618144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa6696-1075-46bc-8889-b3ac08821377 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:19.619946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa6696-1075-46bc-8889-b3ac08821377 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:19.655307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 510/1027] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:36:19 2022] POST /identity/v3/projects => generated 311 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:36:20.939383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4656056-fc83-4cf4-a936-d1ba61974a9b None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:20.939916 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4656056-fc83-4cf4-a936-d1ba61974a9b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:20.939916 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4656056-fc83-4cf4-a936-d1ba61974a9b None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:20.941262 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 518/1028] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:20.946718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a3fc4-23e0-43d6-a34b-00cd1e5b6d9c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:20.947249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a3fc4-23e0-43d6-a34b-00cd1e5b6d9c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:20.947343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a3fc4-23e0-43d6-a34b-00cd1e5b6d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:20.965961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-7e9a3fc4-23e0-43d6-a34b-00cd1e5b6d9c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:20.967143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7e9a3fc4-23e0-43d6-a34b-00cd1e5b6d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:21.003899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:21.041399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 511/1029] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:21.316825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4656056-fc83-4cf4-a936-d1ba61974a9b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:21.346447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:21.425169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4656056-fc83-4cf4-a936-d1ba61974a9b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:21.448589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550643d5-32f4-4c5e-99ad-a31f1ed5e46c None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:21.448869 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550643d5-32f4-4c5e-99ad-a31f1ed5e46c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:21.449139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550643d5-32f4-4c5e-99ad-a31f1ed5e46c None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:21.450595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 512/1030] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:21.455489 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6dadf7-19ae-4a94-bf22-3ae8c7a7f091 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:21.455772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6dadf7-19ae-4a94-bf22-3ae8c7a7f091 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:21.456041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6dadf7-19ae-4a94-bf22-3ae8c7a7f091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:21.465646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eafe7-f1cd-4be7-8072-69862b7f0887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:21.466289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eafe7-f1cd-4be7-8072-69862b7f0887 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:21.466571 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eafe7-f1cd-4be7-8072-69862b7f0887 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:21.466842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eafe7-f1cd-4be7-8072-69862b7f0887 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:21.476145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-bc6dadf7-19ae-4a94-bf22-3ae8c7a7f091 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:21.477684 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-bc6dadf7-19ae-4a94-bf22-3ae8c7a7f091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:21.478259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888eafe7-f1cd-4be7-8072-69862b7f0887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-888eafe7-f1cd-4be7-8072-69862b7f0887 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:21.483169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:21.484117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:21.484962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 519/1031] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Aug 31 09:36:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:21.490182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eafe7-f1cd-4be7-8072-69862b7f0887 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:21.556254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 513/1032] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:21.587282 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:21.599949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eafe7-f1cd-4be7-8072-69862b7f0887 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:21.631011 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d167014-fa9c-47b1-a973-b921252b6171 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:21.631346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d167014-fa9c-47b1-a973-b921252b6171 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:21.631575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d167014-fa9c-47b1-a973-b921252b6171 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:21.633649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 514/1033] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:36:21.635894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf92509-6135-4a24-a49d-05292fbd7bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:21.636502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf92509-6135-4a24-a49d-05292fbd7bf7 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:21.636778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf92509-6135-4a24-a49d-05292fbd7bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:21.637048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf92509-6135-4a24-a49d-05292fbd7bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:21.637713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf92509-6135-4a24-a49d-05292fbd7bf7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:21.638012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf92509-6135-4a24-a49d-05292fbd7bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:21.638635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f3e96-4f33-4ca1-a728-7fcb4385da8f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:21.638919 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f3e96-4f33-4ca1-a728-7fcb4385da8f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:21.639478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f3e96-4f33-4ca1-a728-7fcb4385da8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:21.639662 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf92509-6135-4a24-a49d-05292fbd7bf7 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:21.653362 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 520/1034] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 09:36:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:21.654061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf92509-6135-4a24-a49d-05292fbd7bf7 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:21.659813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-964f3e96-4f33-4ca1-a728-7fcb4385da8f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:21.660991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-964f3e96-4f33-4ca1-a728-7fcb4385da8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:21.732435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 515/1035] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:21.770253 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf92509-6135-4a24-a49d-05292fbd7bf7 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:21.807050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce14e7-0cc0-46ac-adb1-fed4251e6250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:21.807711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce14e7-0cc0-46ac-adb1-fed4251e6250 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:21.807982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce14e7-0cc0-46ac-adb1-fed4251e6250 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:21.808248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce14e7-0cc0-46ac-adb1-fed4251e6250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:21.819358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce14e7-0cc0-46ac-adb1-fed4251e6250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:21.821434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce14e7-0cc0-46ac-adb1-fed4251e6250 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:21.829623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f3e96-4f33-4ca1-a728-7fcb4385da8f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-d0ce14e7-0cc0-46ac-adb1-fed4251e6250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:21.836779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:21.838151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:21.839468 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:21.839468 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:21.839468 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:21.842994 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 521/1036] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:21.843702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce14e7-0cc0-46ac-adb1-fed4251e6250 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:21.846579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:21.867058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:21.945215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f3e96-4f33-4ca1-a728-7fcb4385da8f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:21.959107 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce14e7-0cc0-46ac-adb1-fed4251e6250 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:21.982675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9c5f5-0cf1-471b-ab5b-e76d4255051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:21.983562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9c5f5-0cf1-471b-ab5b-e76d4255051f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:21.983844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9c5f5-0cf1-471b-ab5b-e76d4255051f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:21.984118 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9c5f5-0cf1-471b-ab5b-e76d4255051f None admin] PATH_INFO: `/v3/services/image` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:21.984704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9c5f5-0cf1-471b-ab5b-e76d4255051f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:21.986079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9c5f5-0cf1-471b-ab5b-e76d4255051f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-8ab9c5f5-0cf1-471b-ab5b-e76d4255051f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:21.995926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:21.996915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:21.997767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 516/1037] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Aug 31 09:36:21 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:22.003028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9c5f5-0cf1-471b-ab5b-e76d4255051f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:22.003890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e09b9e-f56c-4322-96ce-e079d6524219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:22.004587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e09b9e-f56c-4322-96ce-e079d6524219 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:22.004783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e09b9e-f56c-4322-96ce-e079d6524219 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:22.005063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e09b9e-f56c-4322-96ce-e079d6524219 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:22.005805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e09b9e-f56c-4322-96ce-e079d6524219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:22.006010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e09b9e-f56c-4322-96ce-e079d6524219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:22.007413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e09b9e-f56c-4322-96ce-e079d6524219 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:22.021338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 522/1038] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:22.022158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e09b9e-f56c-4322-96ce-e079d6524219 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:22.093713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:22.107073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:22.113827 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9c5f5-0cf1-471b-ab5b-e76d4255051f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:22.135208 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e09b9e-f56c-4322-96ce-e079d6524219 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:22.151741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f1b43-8287-4a91-8abd-4d6248cd9a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:22.152345 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f1b43-8287-4a91-8abd-4d6248cd9a72 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:22.152616 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f1b43-8287-4a91-8abd-4d6248cd9a72 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:22.152881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f1b43-8287-4a91-8abd-4d6248cd9a72 None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:22.153498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f1b43-8287-4a91-8abd-4d6248cd9a72 None admin] RBAC: Adding query filter params (name=image) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:22.153795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f1b43-8287-4a91-8abd-4d6248cd9a72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:22.155187 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f1b43-8287-4a91-8abd-4d6248cd9a72 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:22.160256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 517/1039] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:22.161047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f1b43-8287-4a91-8abd-4d6248cd9a72 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:22.176148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbae920-6693-4060-9477-acabc1a45228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:22.176786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbae920-6693-4060-9477-acabc1a45228 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:22.177060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbae920-6693-4060-9477-acabc1a45228 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:22.177329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbae920-6693-4060-9477-acabc1a45228 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:22.178463 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbae920-6693-4060-9477-acabc1a45228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:22.179913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbae920-6693-4060-9477-acabc1a45228 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:22.181743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 523/1040] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:22.182308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbae920-6693-4060-9477-acabc1a45228 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:22.271283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f1b43-8287-4a91-8abd-4d6248cd9a72 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:22.290421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbae920-6693-4060-9477-acabc1a45228 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:22.308013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed1cb4-f5d2-4862-ab01-dcef2e4c25c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:22.308716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed1cb4-f5d2-4862-ab01-dcef2e4c25c8 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:22.308964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed1cb4-f5d2-4862-ab01-dcef2e4c25c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:22.309162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed1cb4-f5d2-4862-ab01-dcef2e4c25c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:22.319985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:22.324923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed1cb4-f5d2-4862-ab01-dcef2e4c25c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:22.326372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed1cb4-f5d2-4862-ab01-dcef2e4c25c8 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:22.327209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6032de-7f63-4a8d-9bef-809e47fc82d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:22.327822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6032de-7f63-4a8d-9bef-809e47fc82d2 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:22.328110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6032de-7f63-4a8d-9bef-809e47fc82d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:22.328387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6032de-7f63-4a8d-9bef-809e47fc82d2 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:22.329048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6032de-7f63-4a8d-9bef-809e47fc82d2 None admin] RBAC: Adding query filter params (type=image) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:22.329361 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6032de-7f63-4a8d-9bef-809e47fc82d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:22.330727 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6032de-7f63-4a8d-9bef-809e47fc82d2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:22.335862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 524/1041] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:22.337514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6032de-7f63-4a8d-9bef-809e47fc82d2 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-45ed1cb4-f5d2-4862-ab01-dcef2e4c25c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:36:22.339349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:22.340290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:22.341603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:22.341603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:22.341603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:36:22.341603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:36:22.341603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:36:22.341603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:36:22.341603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:36:22.341603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:36:22.341603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:36:22.341603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:22.341603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 518/1042] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 09:36:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:22.343773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed1cb4-f5d2-4862-ab01-dcef2e4c25c8 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:22.369604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:22.447806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6032de-7f63-4a8d-9bef-809e47fc82d2 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:22.454483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed1cb4-f5d2-4862-ab01-dcef2e4c25c8 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:22.484713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e3141-cbfc-456a-bebc-83f5cf1f9880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:22.485330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e3141-cbfc-456a-bebc-83f5cf1f9880 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:22.485612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e3141-cbfc-456a-bebc-83f5cf1f9880 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:22.485887 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e3141-cbfc-456a-bebc-83f5cf1f9880 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:22.486547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e3141-cbfc-456a-bebc-83f5cf1f9880 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:22.487999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e3141-cbfc-456a-bebc-83f5cf1f9880 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:22.491720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505350e-53d7-4534-a4cc-6505ea49574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:22.492328 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505350e-53d7-4534-a4cc-6505ea49574b None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:22.492650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505350e-53d7-4534-a4cc-6505ea49574b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:22.492922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505350e-53d7-4534-a4cc-6505ea49574b None admin] PATH_INFO: `/v3/endpoints` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:22.493581 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505350e-53d7-4534-a4cc-6505ea49574b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:22.494932 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505350e-53d7-4534-a4cc-6505ea49574b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:22.511280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 525/1043] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:36:22 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:22.512253 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e3141-cbfc-456a-bebc-83f5cf1f9880 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:22.520539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 519/1044] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:36:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:36:22.581416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:22.654655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e3141-cbfc-456a-bebc-83f5cf1f9880 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:22.704548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372039fd-34cd-4564-9a1a-435ac2ef029c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:22.705168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372039fd-34cd-4564-9a1a-435ac2ef029c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:22.705445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372039fd-34cd-4564-9a1a-435ac2ef029c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:22.705796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372039fd-34cd-4564-9a1a-435ac2ef029c None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:22.706449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372039fd-34cd-4564-9a1a-435ac2ef029c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:22.706746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372039fd-34cd-4564-9a1a-435ac2ef029c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:22.708258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372039fd-34cd-4564-9a1a-435ac2ef029c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:22.728450 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 526/1045] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 09:36:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:22.735085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505350e-53d7-4534-a4cc-6505ea49574b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:22.827358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:22.860731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505350e-53d7-4534-a4cc-6505ea49574b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:22.901933 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec65125-324a-4913-88ec-e2c903193e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:22.902557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec65125-324a-4913-88ec-e2c903193e20 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:22.902835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec65125-324a-4913-88ec-e2c903193e20 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:22.903105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec65125-324a-4913-88ec-e2c903193e20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:22.982503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec65125-324a-4913-88ec-e2c903193e20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:22.984028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec65125-324a-4913-88ec-e2c903193e20 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:23.068375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:23.100941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 520/1046] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:36:23.106659 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372039fd-34cd-4564-9a1a-435ac2ef029c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:23.209049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372039fd-34cd-4564-9a1a-435ac2ef029c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:23.211470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:23.246777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f12b44-6ce2-4e4c-aaa6-cfde8dd7752f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:23.247377 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f12b44-6ce2-4e4c-aaa6-cfde8dd7752f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:23.247641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f12b44-6ce2-4e4c-aaa6-cfde8dd7752f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:23.247908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f12b44-6ce2-4e4c-aaa6-cfde8dd7752f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:23.260708 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f12b44-6ce2-4e4c-aaa6-cfde8dd7752f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:23.262171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f12b44-6ce2-4e4c-aaa6-cfde8dd7752f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-89f12b44-6ce2-4e4c-aaa6-cfde8dd7752f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:23.278765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:23.279850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:23.280736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:23.280736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:23.280736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:23.283446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 527/1047] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:23.285611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec65125-324a-4913-88ec-e2c903193e20 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:23.391958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec65125-324a-4913-88ec-e2c903193e20 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:23.427190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf9fbe-90dc-4a8c-a56f-930926a3725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:23.427797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf9fbe-90dc-4a8c-a56f-930926a3725f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:23.428073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf9fbe-90dc-4a8c-a56f-930926a3725f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:23.428340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf9fbe-90dc-4a8c-a56f-930926a3725f None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:23.428976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf9fbe-90dc-4a8c-a56f-930926a3725f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:23.429275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf9fbe-90dc-4a8c-a56f-930926a3725f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:23.431891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf9fbe-90dc-4a8c-a56f-930926a3725f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:23.448019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 521/1048] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:23.449772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f12b44-6ce2-4e4c-aaa6-cfde8dd7752f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:23.553272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f12b44-6ce2-4e4c-aaa6-cfde8dd7752f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:23.589531 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2f479-6857-438b-a85f-45dfa0b1a637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:23.590168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2f479-6857-438b-a85f-45dfa0b1a637 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:23.590445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2f479-6857-438b-a85f-45dfa0b1a637 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:23.590711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2f479-6857-438b-a85f-45dfa0b1a637 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:23.598014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2f479-6857-438b-a85f-45dfa0b1a637 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:23.599479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2f479-6857-438b-a85f-45dfa0b1a637 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-70e2f479-6857-438b-a85f-45dfa0b1a637 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:23.611004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:23.612099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:23.612998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:23.612998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:23.612998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:23.612998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:23.612998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:23.612998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:23.612998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:23.612998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:23.615199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 528/1049] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:23.618565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf9fbe-90dc-4a8c-a56f-930926a3725f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:23.657481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:23.729447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf9fbe-90dc-4a8c-a56f-930926a3725f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:23.765331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43760e-3139-4667-852b-9a84571dd0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:23.765941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43760e-3139-4667-852b-9a84571dd0ba None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:23.766216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43760e-3139-4667-852b-9a84571dd0ba None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:23.766484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43760e-3139-4667-852b-9a84571dd0ba None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:23.767186 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43760e-3139-4667-852b-9a84571dd0ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:23.767492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43760e-3139-4667-852b-9a84571dd0ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:23.768911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43760e-3139-4667-852b-9a84571dd0ba None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:23.787094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 522/1050] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:36:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:23.788560 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2f479-6857-438b-a85f-45dfa0b1a637 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:23.809882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:23.896812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2f479-6857-438b-a85f-45dfa0b1a637 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:23.934595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accaac0b-2a9f-4b8d-a8af-ba725c8d2cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:23.935232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accaac0b-2a9f-4b8d-a8af-ba725c8d2cfc None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:23.935514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accaac0b-2a9f-4b8d-a8af-ba725c8d2cfc None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:23.935786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accaac0b-2a9f-4b8d-a8af-ba725c8d2cfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:23.936508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accaac0b-2a9f-4b8d-a8af-ba725c8d2cfc None admin] RBAC: Adding query filter params (role.id=5bb2daf5cef542028e8802f2f06a46bd, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=ef72bb95992c4a21ae2dbb2a08c51421) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:23.936816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accaac0b-2a9f-4b8d-a8af-ba725c8d2cfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:23.938236 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accaac0b-2a9f-4b8d-a8af-ba725c8d2cfc None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:23.950144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 529/1051] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:23 2022] GET /identity/v3/role_assignments?user.id=ef72bb95992c4a21ae2dbb2a08c51421&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=5bb2daf5cef542028e8802f2f06a46bd => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:24.504306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70a919-fc72-4111-a856-102c725178db None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:24.504839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70a919-fc72-4111-a856-102c725178db None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:24.504839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70a919-fc72-4111-a856-102c725178db None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:24.506306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 523/1052] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:24.511869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53185f44-c8e2-42f2-8c72-97517e393129 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:24.512322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53185f44-c8e2-42f2-8c72-97517e393129 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:24.512400 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53185f44-c8e2-42f2-8c72-97517e393129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:24.531822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-53185f44-c8e2-42f2-8c72-97517e393129 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:24.533021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-53185f44-c8e2-42f2-8c72-97517e393129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:24.582071 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba421a-c025-4007-8755-c10c56b1bc75 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:24.582357 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba421a-c025-4007-8755-c10c56b1bc75 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:24.582633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba421a-c025-4007-8755-c10c56b1bc75 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:24.584118 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 524/1053] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:24.589724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444515c-2837-4b20-8295-b7edfc607763 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:24.590007 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444515c-2837-4b20-8295-b7edfc607763 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:24.590285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444515c-2837-4b20-8295-b7edfc607763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:24.603275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 530/1054] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:24.612698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a444515c-2837-4b20-8295-b7edfc607763 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:24.614000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a444515c-2837-4b20-8295-b7edfc607763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:24.685189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 525/1055] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:24.872738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53185f44-c8e2-42f2-8c72-97517e393129 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:24.884696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:24.973712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53185f44-c8e2-42f2-8c72-97517e393129 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:24.975221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444515c-2837-4b20-8295-b7edfc607763 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:25.007018 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:25.013864 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22d79d-37c3-4f71-a914-501a256d432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:25.014748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22d79d-37c3-4f71-a914-501a256d432d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:25.015028 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22d79d-37c3-4f71-a914-501a256d432d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:25.015331 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22d79d-37c3-4f71-a914-501a256d432d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:25.091005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22d79d-37c3-4f71-a914-501a256d432d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:25.092499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22d79d-37c3-4f71-a914-501a256d432d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:25.098546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444515c-2837-4b20-8295-b7edfc607763 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:25.105050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:25.138883 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ff94e-53a9-435f-bf91-256e7853dc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:25.139715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ff94e-53a9-435f-bf91-256e7853dc92 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:25.139992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ff94e-53a9-435f-bf91-256e7853dc92 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:25.140269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ff94e-53a9-435f-bf91-256e7853dc92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:25.151956 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ff94e-53a9-435f-bf91-256e7853dc92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-3a3ff94e-53a9-435f-bf91-256e7853dc92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:25.156671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:25.157843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:25.158674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 526/1056] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:25.163872 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ff94e-53a9-435f-bf91-256e7853dc92 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:25.209884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 531/1057] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:36:25.216304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22d79d-37c3-4f71-a914-501a256d432d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:25.277380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:25.297280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ff94e-53a9-435f-bf91-256e7853dc92 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:25.333928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22d79d-37c3-4f71-a914-501a256d432d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:25.346967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6d5af-6b72-48a4-b523-db06b0825b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:25.347637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6d5af-6b72-48a4-b523-db06b0825b61 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:25.347915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6d5af-6b72-48a4-b523-db06b0825b61 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:25.348188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6d5af-6b72-48a4-b523-db06b0825b61 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:25.348873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6d5af-6b72-48a4-b523-db06b0825b61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:25.349177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6d5af-6b72-48a4-b523-db06b0825b61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:25.350571 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:25.350703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6d5af-6b72-48a4-b523-db06b0825b61 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:25.360198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 527/1058] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:25.366633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6d5af-6b72-48a4-b523-db06b0825b61 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:25.388417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9331f-d6f9-4f20-a56f-ca83af63548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:25.389387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9331f-d6f9-4f20-a56f-ca83af63548f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:25.389615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9331f-d6f9-4f20-a56f-ca83af63548f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:25.389889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9331f-d6f9-4f20-a56f-ca83af63548f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:25.401942 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9331f-d6f9-4f20-a56f-ca83af63548f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:25.403505 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9331f-d6f9-4f20-a56f-ca83af63548f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-9dd9331f-d6f9-4f20-a56f-ca83af63548f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:25.419856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:25.421063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:25.421966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:25.421966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:25.421966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:25.421966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 532/1059] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:25.427337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9331f-d6f9-4f20-a56f-ca83af63548f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:25.493478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6d5af-6b72-48a4-b523-db06b0825b61 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:25.534691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601604c-0dba-45eb-9aff-d90e057c76b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:25.535460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601604c-0dba-45eb-9aff-d90e057c76b2 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:25.535817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601604c-0dba-45eb-9aff-d90e057c76b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:25.536098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601604c-0dba-45eb-9aff-d90e057c76b2 None admin] PATH_INFO: `/v3/users/1937dfec03b74d76a29ee52e482119e9` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:25.545358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9331f-d6f9-4f20-a56f-ca83af63548f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:25.554978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601604c-0dba-45eb-9aff-d90e057c76b2 None admin] RBAC: Authorizing `identity:get_user(user_id=1937dfec03b74d76a29ee52e482119e9)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:25.556467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601604c-0dba-45eb-9aff-d90e057c76b2 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:25.558309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 528/1060] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:25 2022] GET /identity/v3/users/1937dfec03b74d76a29ee52e482119e9 => generated 285 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:25.564189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601604c-0dba-45eb-9aff-d90e057c76b2 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:25.568950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:25.591834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d94e6-4ea1-4019-bbc3-c091d15431f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:25.592449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d94e6-4ea1-4019-bbc3-c091d15431f2 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:25.592736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d94e6-4ea1-4019-bbc3-c091d15431f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:25.593011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d94e6-4ea1-4019-bbc3-c091d15431f2 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:25.593659 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d94e6-4ea1-4019-bbc3-c091d15431f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:25.593963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d94e6-4ea1-4019-bbc3-c091d15431f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:25.595374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d94e6-4ea1-4019-bbc3-c091d15431f2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:25.609074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 533/1061] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:25.679321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601604c-0dba-45eb-9aff-d90e057c76b2 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:25.718789 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91504d18-1a7a-455b-8bfd-dcf78b8f701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:25.719455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91504d18-1a7a-455b-8bfd-dcf78b8f701c None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:25.719735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91504d18-1a7a-455b-8bfd-dcf78b8f701c None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:25.720008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91504d18-1a7a-455b-8bfd-dcf78b8f701c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:25.795206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91504d18-1a7a-455b-8bfd-dcf78b8f701c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:25.796705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91504d18-1a7a-455b-8bfd-dcf78b8f701c None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:25.799540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:25.861347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7393-80bf-4d82-84f5-bf8f2a7e8bf7 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:25.861625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7393-80bf-4d82-84f5-bf8f2a7e8bf7 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:25.861897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7393-80bf-4d82-84f5-bf8f2a7e8bf7 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:25.867909 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 534/1062] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:25 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:36:25.869232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e88faf-3cc1-40dd-a617-6ba3d4a8c7a0 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:25.869524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e88faf-3cc1-40dd-a617-6ba3d4a8c7a0 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:25.869799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e88faf-3cc1-40dd-a617-6ba3d4a8c7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:25.891347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-25e88faf-3cc1-40dd-a617-6ba3d4a8c7a0 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:25.892897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-25e88faf-3cc1-40dd-a617-6ba3d4a8c7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:25.911069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 529/1063] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:36:25.916046 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:25.917463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91504d18-1a7a-455b-8bfd-dcf78b8f701c None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:25.971984 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 535/1064] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:26.020757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91504d18-1a7a-455b-8bfd-dcf78b8f701c None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:26.023063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:26.055097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ebfc5-0e9c-401d-bd45-f0a814099c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:26.055941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ebfc5-0e9c-401d-bd45-f0a814099c2d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:26.056222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ebfc5-0e9c-401d-bd45-f0a814099c2d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:26.056494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ebfc5-0e9c-401d-bd45-f0a814099c2d None admin] PATH_INFO: `/v3/projects/140b555fb3954bd98f1c35f4297ffbc0` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:26.057637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ebfc5-0e9c-401d-bd45-f0a814099c2d None admin] RBAC: Authorizing `identity:get_project(project_id=140b555fb3954bd98f1c35f4297ffbc0)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:26.059096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ebfc5-0e9c-401d-bd45-f0a814099c2d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:26.060752 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 530/1065] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:25 2022] GET /identity/v3/projects/140b555fb3954bd98f1c35f4297ffbc0 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:26.066728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e88faf-3cc1-40dd-a617-6ba3d4a8c7a0 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:26.168825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e88faf-3cc1-40dd-a617-6ba3d4a8c7a0 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:26.202851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6182a-66df-43e8-8be9-1c82eb8b52ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:26.203476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6182a-66df-43e8-8be9-1c82eb8b52ef None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:26.203755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6182a-66df-43e8-8be9-1c82eb8b52ef None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:26.204023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6182a-66df-43e8-8be9-1c82eb8b52ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:26.204741 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6182a-66df-43e8-8be9-1c82eb8b52ef None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=140b555fb3954bd98f1c35f4297ffbc0, user.id=1937dfec03b74d76a29ee52e482119e9) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:26.205048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6182a-66df-43e8-8be9-1c82eb8b52ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:26.206459 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6182a-66df-43e8-8be9-1c82eb8b52ef None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:26.212286 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 536/1066] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:26 2022] GET /identity/v3/role_assignments?user.id=1937dfec03b74d76a29ee52e482119e9&scope.project.id=140b555fb3954bd98f1c35f4297ffbc0&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:26.245892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ebfc5-0e9c-401d-bd45-f0a814099c2d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:26.353017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ebfc5-0e9c-401d-bd45-f0a814099c2d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:26.390342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e5a5d-fb7d-4fd3-b3f1-bb7a26eb0127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:26.390967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e5a5d-fb7d-4fd3-b3f1-bb7a26eb0127 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:26.391268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e5a5d-fb7d-4fd3-b3f1-bb7a26eb0127 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:26.391536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e5a5d-fb7d-4fd3-b3f1-bb7a26eb0127 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:26.401799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e5a5d-fb7d-4fd3-b3f1-bb7a26eb0127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-5e8e5a5d-fb7d-4fd3-b3f1-bb7a26eb0127 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:26.406493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:26.407426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:26.408260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 531/1067] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Aug 31 09:36:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:26.413987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6182a-66df-43e8-8be9-1c82eb8b52ef None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:26.432959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:26.516852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6182a-66df-43e8-8be9-1c82eb8b52ef None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:26.549731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7fd76-bd6b-45b3-ac37-b81b354a34f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:26.550380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7fd76-bd6b-45b3-ac37-b81b354a34f0 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:26.550592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7fd76-bd6b-45b3-ac37-b81b354a34f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:26.550858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7fd76-bd6b-45b3-ac37-b81b354a34f0 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:26.551544 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7fd76-bd6b-45b3-ac37-b81b354a34f0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:26.551842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7fd76-bd6b-45b3-ac37-b81b354a34f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:26.553186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7fd76-bd6b-45b3-ac37-b81b354a34f0 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:26.561229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 537/1068] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 09:36:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:26.566949 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e5a5d-fb7d-4fd3-b3f1-bb7a26eb0127 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:26.598623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:26.667717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e5a5d-fb7d-4fd3-b3f1-bb7a26eb0127 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:26.701140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d4d08-2916-4e2e-8ddf-9bec150e1f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:26.701760 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d4d08-2916-4e2e-8ddf-9bec150e1f5a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:26.702031 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d4d08-2916-4e2e-8ddf-9bec150e1f5a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:26.702292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d4d08-2916-4e2e-8ddf-9bec150e1f5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:26.712779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d4d08-2916-4e2e-8ddf-9bec150e1f5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:26.714219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d4d08-2916-4e2e-8ddf-9bec150e1f5a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-5b1d4d08-2916-4e2e-8ddf-9bec150e1f5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:26.728399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:26.729467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:26.730366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:26.730366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:26.730366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:26.730366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 532/1069] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:26.735340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7fd76-bd6b-45b3-ac37-b81b354a34f0 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:26.813156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:26.834790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7fd76-bd6b-45b3-ac37-b81b354a34f0 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:26.869055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd4277-97b1-4a47-87b5-db9b40563627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:26.869764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd4277-97b1-4a47-87b5-db9b40563627 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:26.870114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd4277-97b1-4a47-87b5-db9b40563627 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:26.870451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd4277-97b1-4a47-87b5-db9b40563627 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:26.871109 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd4277-97b1-4a47-87b5-db9b40563627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:26.871433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd4277-97b1-4a47-87b5-db9b40563627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:26.872806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd4277-97b1-4a47-87b5-db9b40563627 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:26.885476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 538/1070] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:26.891376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d4d08-2916-4e2e-8ddf-9bec150e1f5a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:26.974730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:26.991217 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d4d08-2916-4e2e-8ddf-9bec150e1f5a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:27.025647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9ec14-1b8e-426c-ba4b-78e4c7ab73b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:27.026271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9ec14-1b8e-426c-ba4b-78e4c7ab73b7 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:27.026556 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9ec14-1b8e-426c-ba4b-78e4c7ab73b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:27.026828 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9ec14-1b8e-426c-ba4b-78e4c7ab73b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:27.034281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9ec14-1b8e-426c-ba4b-78e4c7ab73b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:27.035935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9ec14-1b8e-426c-ba4b-78e4c7ab73b7 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-b2d9ec14-1b8e-426c-ba4b-78e4c7ab73b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:36:27.047249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:27.048275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:27.049794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:27.049794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:27.049794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:36:27.049794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:36:27.049794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:36:27.049794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:36:27.049794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:36:27.049794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:36:27.049794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:36:27.049794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:27.049794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 533/1071] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 09:36:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:27.054176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd4277-97b1-4a47-87b5-db9b40563627 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:27.173740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd4277-97b1-4a47-87b5-db9b40563627 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:27.211878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a519231-aeda-4602-938e-e98676e59a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:27.212822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a519231-aeda-4602-938e-e98676e59a94 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:27.213103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a519231-aeda-4602-938e-e98676e59a94 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:27.213374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a519231-aeda-4602-938e-e98676e59a94 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:27.214097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a519231-aeda-4602-938e-e98676e59a94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:27.214398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a519231-aeda-4602-938e-e98676e59a94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:27.215863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a519231-aeda-4602-938e-e98676e59a94 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:27.224205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:27.237932 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 539/1072] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 09:36:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:27.244917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9ec14-1b8e-426c-ba4b-78e4c7ab73b7 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:27.347969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9ec14-1b8e-426c-ba4b-78e4c7ab73b7 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:27.380226 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bb247-88f2-43ff-ba17-c85e8d2224ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:27.380847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bb247-88f2-43ff-ba17-c85e8d2224ef None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:27.381115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bb247-88f2-43ff-ba17-c85e8d2224ef None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:27.381379 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bb247-88f2-43ff-ba17-c85e8d2224ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:27.395184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bb247-88f2-43ff-ba17-c85e8d2224ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:27.396659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bb247-88f2-43ff-ba17-c85e8d2224ef None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:27.399975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-a00bb247-88f2-43ff-ba17-c85e8d2224ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:27.413648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:27.414756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:27.415772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:27.415772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:27.415772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:27.415772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 534/1073] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:27.421389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a519231-aeda-4602-938e-e98676e59a94 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:27.522171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a519231-aeda-4602-938e-e98676e59a94 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:27.556486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705c749-8dc0-4224-8d71-3255f2466e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:27.557143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705c749-8dc0-4224-8d71-3255f2466e0a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:27.557350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705c749-8dc0-4224-8d71-3255f2466e0a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:27.557621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705c749-8dc0-4224-8d71-3255f2466e0a None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:27.558255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705c749-8dc0-4224-8d71-3255f2466e0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:27.558552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705c749-8dc0-4224-8d71-3255f2466e0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:27.559939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705c749-8dc0-4224-8d71-3255f2466e0a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:27.576290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 540/1074] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:27.577624 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bb247-88f2-43ff-ba17-c85e8d2224ef None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:27.679356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bb247-88f2-43ff-ba17-c85e8d2224ef None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:27.713034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dec45-a7bb-4231-bd5b-50dc57243bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:27.713651 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dec45-a7bb-4231-bd5b-50dc57243bda None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:27.713924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dec45-a7bb-4231-bd5b-50dc57243bda None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:27.714193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dec45-a7bb-4231-bd5b-50dc57243bda None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:27.720783 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63dec45-a7bb-4231-bd5b-50dc57243bda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:27.722223 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63dec45-a7bb-4231-bd5b-50dc57243bda None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-c63dec45-a7bb-4231-bd5b-50dc57243bda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:27.733923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:27.734962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:27.736378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:27.736378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:27.736378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:27.736378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:27.736378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:27.736378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:27.736378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:27.736378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:27.736378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 535/1075] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:27.740838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705c749-8dc0-4224-8d71-3255f2466e0a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:27.799578 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:27.845160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705c749-8dc0-4224-8d71-3255f2466e0a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:27.884471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f755405-4d24-4452-9bb7-78248d3d7902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:27.885252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f755405-4d24-4452-9bb7-78248d3d7902 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:27.885370 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f755405-4d24-4452-9bb7-78248d3d7902 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:27.885626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f755405-4d24-4452-9bb7-78248d3d7902 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:27.886431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f755405-4d24-4452-9bb7-78248d3d7902 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:27.886733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f755405-4d24-4452-9bb7-78248d3d7902 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:27.888194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f755405-4d24-4452-9bb7-78248d3d7902 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:27.902249 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 541/1076] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:36:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:27.908247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dec45-a7bb-4231-bd5b-50dc57243bda None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:27.966269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:28.015543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dec45-a7bb-4231-bd5b-50dc57243bda None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:28.049765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcd1fa-6627-4544-8e55-6671cb1a7f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:28.050421 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcd1fa-6627-4544-8e55-6671cb1a7f42 None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:28.050696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcd1fa-6627-4544-8e55-6671cb1a7f42 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:28.050965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcd1fa-6627-4544-8e55-6671cb1a7f42 None admin] PATH_INFO: `/v3/projects/8ef8b586382941408cd88ee654c42d32/users/ef72bb95992c4a21ae2dbb2a08c51421/roles/5bb2daf5cef542028e8802f2f06a46bd` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:28.053128 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcd1fa-6627-4544-8e55-6671cb1a7f42 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef8b586382941408cd88ee654c42d32, user_id=ef72bb95992c4a21ae2dbb2a08c51421, role_id=5bb2daf5cef542028e8802f2f06a46bd)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:28.054565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcd1fa-6627-4544-8e55-6671cb1a7f42 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:28.068860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 536/1077] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:36:27 2022] PUT /identity/v3/projects/8ef8b586382941408cd88ee654c42d32/users/ef72bb95992c4a21ae2dbb2a08c51421/roles/5bb2daf5cef542028e8802f2f06a46bd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:36:28.145698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af7a77-b478-4ee6-ace4-28e13fa70b5f None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:28.145985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af7a77-b478-4ee6-ace4-28e13fa70b5f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:28.146262 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af7a77-b478-4ee6-ace4-28e13fa70b5f None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:28.147748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 542/1078] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:28.155188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a0f14-687c-4306-b013-878067bdbf0b None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:28.155483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a0f14-687c-4306-b013-878067bdbf0b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:28.155757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a0f14-687c-4306-b013-878067bdbf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:28.176027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-496a0f14-687c-4306-b013-878067bdbf0b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:28.177280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-496a0f14-687c-4306-b013-878067bdbf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:28.257249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 537/1079] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:28.528878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af7a77-b478-4ee6-ace4-28e13fa70b5f None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:28.630798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af7a77-b478-4ee6-ace4-28e13fa70b5f None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:28.647584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:28.666117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cf231-600f-4b30-a24d-8c90646d0432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:28.666754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cf231-600f-4b30-a24d-8c90646d0432 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:28.667033 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cf231-600f-4b30-a24d-8c90646d0432 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:28.667333 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cf231-600f-4b30-a24d-8c90646d0432 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:28.677321 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cf231-600f-4b30-a24d-8c90646d0432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-321cf231-600f-4b30-a24d-8c90646d0432 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:28.682120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:28.683141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:28.684146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 543/1080] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:28.689270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a0f14-687c-4306-b013-878067bdbf0b None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:28.734866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:28.791735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a0f14-687c-4306-b013-878067bdbf0b None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:28.825770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f419c-92e6-42ad-9f9e-1d51a735e3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:28.826370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f419c-92e6-42ad-9f9e-1d51a735e3c3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:28.826648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f419c-92e6-42ad-9f9e-1d51a735e3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:28.826917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f419c-92e6-42ad-9f9e-1d51a735e3c3 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:28.827595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f419c-92e6-42ad-9f9e-1d51a735e3c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:28.827894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f419c-92e6-42ad-9f9e-1d51a735e3c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:28.829260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f419c-92e6-42ad-9f9e-1d51a735e3c3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:28.837677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 538/1081] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:28.843578 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cf231-600f-4b30-a24d-8c90646d0432 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:28.945430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cf231-600f-4b30-a24d-8c90646d0432 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:28.979419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08564c15-168c-4169-a08c-e268f9d72db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:28.980061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08564c15-168c-4169-a08c-e268f9d72db1 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:28.980340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08564c15-168c-4169-a08c-e268f9d72db1 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:28.980614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08564c15-168c-4169-a08c-e268f9d72db1 None admin] PATH_INFO: `/v3/users/1937dfec03b74d76a29ee52e482119e9` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:28.981762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08564c15-168c-4169-a08c-e268f9d72db1 None admin] RBAC: Authorizing `identity:get_user(user_id=1937dfec03b74d76a29ee52e482119e9)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:28.983235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08564c15-168c-4169-a08c-e268f9d72db1 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:28.984843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 544/1082] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:28 2022] GET /identity/v3/users/1937dfec03b74d76a29ee52e482119e9 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:28.990613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f419c-92e6-42ad-9f9e-1d51a735e3c3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:29.093200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f419c-92e6-42ad-9f9e-1d51a735e3c3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:29.103414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:29.129980 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fc0da-ce5a-456d-8a45-185c719adc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:29.130634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fc0da-ce5a-456d-8a45-185c719adc81 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:29.130912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fc0da-ce5a-456d-8a45-185c719adc81 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:29.131214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fc0da-ce5a-456d-8a45-185c719adc81 None admin] PATH_INFO: `/v3/projects/140b555fb3954bd98f1c35f4297ffbc0` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:29.132333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fc0da-ce5a-456d-8a45-185c719adc81 None admin] RBAC: Authorizing `identity:get_project(project_id=140b555fb3954bd98f1c35f4297ffbc0)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:29.134304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fc0da-ce5a-456d-8a45-185c719adc81 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:29.135989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 539/1083] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:28 2022] GET /identity/v3/projects/140b555fb3954bd98f1c35f4297ffbc0 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:29.142139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08564c15-168c-4169-a08c-e268f9d72db1 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:29.190715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:29.247941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08564c15-168c-4169-a08c-e268f9d72db1 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:29.282234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5476e7-99de-4697-aaea-685d558a96fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:29.282888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5476e7-99de-4697-aaea-685d558a96fa None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:29.283196 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5476e7-99de-4697-aaea-685d558a96fa None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:29.283477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5476e7-99de-4697-aaea-685d558a96fa None admin] PATH_INFO: `/v3/projects/140b555fb3954bd98f1c35f4297ffbc0/users/1937dfec03b74d76a29ee52e482119e9/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:29.285391 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5476e7-99de-4697-aaea-685d558a96fa None admin] RBAC: Authorizing `identity:create_grant(project_id=140b555fb3954bd98f1c35f4297ffbc0, user_id=1937dfec03b74d76a29ee52e482119e9, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:29.286815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5476e7-99de-4697-aaea-685d558a96fa None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:29.299069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 545/1084] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:36:29 2022] PUT /identity/v3/projects/140b555fb3954bd98f1c35f4297ffbc0/users/1937dfec03b74d76a29ee52e482119e9/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:36:30.015615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a5573-b91f-4e49-80ee-3d1ac899a333 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:30.016147 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a5573-b91f-4e49-80ee-3d1ac899a333 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:30.016147 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a5573-b91f-4e49-80ee-3d1ac899a333 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:30.017455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 540/1085] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:30.022682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7590fe7-1bde-419b-9ba8-d5ce12883792 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:30.023148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7590fe7-1bde-419b-9ba8-d5ce12883792 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:30.023259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7590fe7-1bde-419b-9ba8-d5ce12883792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:30.041200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-a7590fe7-1bde-419b-9ba8-d5ce12883792 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:30.042407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a7590fe7-1bde-419b-9ba8-d5ce12883792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:30.116238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 546/1086] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:30.388680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a5573-b91f-4e49-80ee-3d1ac899a333 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:30.491640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a5573-b91f-4e49-80ee-3d1ac899a333 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:30.526854 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff9921-659a-418c-b7fa-06f2b234e171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:30.527513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff9921-659a-418c-b7fa-06f2b234e171 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:30.527787 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff9921-659a-418c-b7fa-06f2b234e171 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:30.528056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff9921-659a-418c-b7fa-06f2b234e171 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:30.539478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff9921-659a-418c-b7fa-06f2b234e171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-d7ff9921-659a-418c-b7fa-06f2b234e171 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:30.545297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:30.546575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:30.547626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:30.547626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:30.547626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:30.547626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:30.547626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:30.547626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:30.547626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:30.547626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:30.547626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:30.547626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:30.547626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:36:30.547626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:30.551296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 541/1087] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Aug 31 09:36:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:30.556406 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7590fe7-1bde-419b-9ba8-d5ce12883792 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:30.596863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:30.665695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7590fe7-1bde-419b-9ba8-d5ce12883792 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:30.700482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e1963-764e-4bde-8ddb-58374c73642c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:30.701095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e1963-764e-4bde-8ddb-58374c73642c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:30.701373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e1963-764e-4bde-8ddb-58374c73642c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:30.701647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e1963-764e-4bde-8ddb-58374c73642c None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:30.702321 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e1963-764e-4bde-8ddb-58374c73642c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:30.702619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e1963-764e-4bde-8ddb-58374c73642c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:30.704023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e1963-764e-4bde-8ddb-58374c73642c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:30.713341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 547/1088] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 09:36:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:30.719287 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff9921-659a-418c-b7fa-06f2b234e171 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:30.755530 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:30.829400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff9921-659a-418c-b7fa-06f2b234e171 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:30.866494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899e7c1-d98d-41df-84b2-41dc2b21e4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:30.867714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899e7c1-d98d-41df-84b2-41dc2b21e4ee None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:30.868709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899e7c1-d98d-41df-84b2-41dc2b21e4ee None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:30.868992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899e7c1-d98d-41df-84b2-41dc2b21e4ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:30.881415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899e7c1-d98d-41df-84b2-41dc2b21e4ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:30.883665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899e7c1-d98d-41df-84b2-41dc2b21e4ee None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-9899e7c1-d98d-41df-84b2-41dc2b21e4ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:30.901308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:30.902336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:30.903307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:30.903307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:30.903307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:30.907389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 542/1089] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:30.908395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e1963-764e-4bde-8ddb-58374c73642c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.015451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:31.021875 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e1963-764e-4bde-8ddb-58374c73642c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.055669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22188639-ba5d-4882-9856-e421c419be9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.056312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188639-ba5d-4882-9856-e421c419be9e None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.056566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188639-ba5d-4882-9856-e421c419be9e None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.056816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188639-ba5d-4882-9856-e421c419be9e None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.057455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22188639-ba5d-4882-9856-e421c419be9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:31.057755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22188639-ba5d-4882-9856-e421c419be9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:31.059696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22188639-ba5d-4882-9856-e421c419be9e None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:31.074169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 548/1090] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:31.077906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899e7c1-d98d-41df-84b2-41dc2b21e4ee None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.168003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:31.180453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899e7c1-d98d-41df-84b2-41dc2b21e4ee None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.214195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ac7f0-f5d9-4ab8-a10e-2ca5ab6a9ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.214826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ac7f0-f5d9-4ab8-a10e-2ca5ab6a9ead None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.215104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ac7f0-f5d9-4ab8-a10e-2ca5ab6a9ead None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.215405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ac7f0-f5d9-4ab8-a10e-2ca5ab6a9ead None admin] PATH_INFO: `/v3/users/nova` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.223034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ac7f0-f5d9-4ab8-a10e-2ca5ab6a9ead None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:31.224510 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ac7f0-f5d9-4ab8-a10e-2ca5ab6a9ead None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:31.229275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdf2d8-f899-42a7-b47d-bc118cfcb08b None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.229556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdf2d8-f899-42a7-b47d-bc118cfcb08b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.229937 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdf2d8-f899-42a7-b47d-bc118cfcb08b None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.231374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 549/1091] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-a12ac7f0-f5d9-4ab8-a10e-2ca5ab6a9ead None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:36:31.236761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:31.237900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:31.239233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:31.239233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:31.239233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:36:31.239233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:36:31.239233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:36:31.239233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:36:31.239233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:36:31.239233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:36:31.239233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:36:31.239233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:31.239876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e3952-6ac7-4b51-98bd-bc10d1c80e76 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.239876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e3952-6ac7-4b51-98bd-bc10d1c80e76 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.239876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e3952-6ac7-4b51-98bd-bc10d1c80e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.240073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 543/1092] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:31.244031 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ac7f0-f5d9-4ab8-a10e-2ca5ab6a9ead None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.258661 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-980e3952-6ac7-4b51-98bd-bc10d1c80e76 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:31.259958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-980e3952-6ac7-4b51-98bd-bc10d1c80e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:31.331095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 550/1093] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:31.346007 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ac7f0-f5d9-4ab8-a10e-2ca5ab6a9ead None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.378855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e9bad-afff-4746-8a58-87c5209dd652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.379493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e9bad-afff-4746-8a58-87c5209dd652 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.379771 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e9bad-afff-4746-8a58-87c5209dd652 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.380039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e9bad-afff-4746-8a58-87c5209dd652 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.380752 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e9bad-afff-4746-8a58-87c5209dd652 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:31.381053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e9bad-afff-4746-8a58-87c5209dd652 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:31.382469 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e9bad-afff-4746-8a58-87c5209dd652 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:31.401030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 544/1094] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:31.406735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e3952-6ac7-4b51-98bd-bc10d1c80e76 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.490907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:31.506692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e3952-6ac7-4b51-98bd-bc10d1c80e76 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.541043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3691b8-41e0-4ebb-b400-a27a81311586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.541652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3691b8-41e0-4ebb-b400-a27a81311586 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.541933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3691b8-41e0-4ebb-b400-a27a81311586 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.542203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3691b8-41e0-4ebb-b400-a27a81311586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.599308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e9bad-afff-4746-8a58-87c5209dd652 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.607999 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:31.609227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3691b8-41e0-4ebb-b400-a27a81311586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:31.610756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3691b8-41e0-4ebb-b400-a27a81311586 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:31.694935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:31.702470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e9bad-afff-4746-8a58-87c5209dd652 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.711693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 551/1095] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:36:31.717486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3691b8-41e0-4ebb-b400-a27a81311586 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.739513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca4a60-39be-41b4-9ac9-9f5362056848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.740135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca4a60-39be-41b4-9ac9-9f5362056848 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.740412 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca4a60-39be-41b4-9ac9-9f5362056848 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.740679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca4a60-39be-41b4-9ac9-9f5362056848 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.750985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca4a60-39be-41b4-9ac9-9f5362056848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-03ca4a60-39be-41b4-9ac9-9f5362056848 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:31.755703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:31.756808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:31.758057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 545/1096] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:31.763259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca4a60-39be-41b4-9ac9-9f5362056848 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.822641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3691b8-41e0-4ebb-b400-a27a81311586 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.830481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:31.858608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b87abf-dede-4003-b8d4-2eae281f50a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.859174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b87abf-dede-4003-b8d4-2eae281f50a7 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.859457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b87abf-dede-4003-b8d4-2eae281f50a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.859727 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b87abf-dede-4003-b8d4-2eae281f50a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.866321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca4a60-39be-41b4-9ac9-9f5362056848 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.870139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b87abf-dede-4003-b8d4-2eae281f50a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:31.871611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b87abf-dede-4003-b8d4-2eae281f50a7 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-72b87abf-dede-4003-b8d4-2eae281f50a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:31.887042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:31.888225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:31.889593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:31.889593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:31.889593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:31.891019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 552/1097] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:31.896657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b87abf-dede-4003-b8d4-2eae281f50a7 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.906046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964f6b9-8aeb-40e2-867a-16b71399b54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.906667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964f6b9-8aeb-40e2-867a-16b71399b54d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.906940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964f6b9-8aeb-40e2-867a-16b71399b54d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.907238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964f6b9-8aeb-40e2-867a-16b71399b54d None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.907916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964f6b9-8aeb-40e2-867a-16b71399b54d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:31.908219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964f6b9-8aeb-40e2-867a-16b71399b54d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:31.909635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964f6b9-8aeb-40e2-867a-16b71399b54d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:31.919449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 546/1098] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:31.925290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964f6b9-8aeb-40e2-867a-16b71399b54d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.940142 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:32.002294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b87abf-dede-4003-b8d4-2eae281f50a7 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:32.026265 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964f6b9-8aeb-40e2-867a-16b71399b54d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:32.040597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6336b-5766-4cb3-9c6f-967f4d623cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:32.041206 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6336b-5766-4cb3-9c6f-967f4d623cf2 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.041486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6336b-5766-4cb3-9c6f-967f4d623cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.041762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6336b-5766-4cb3-9c6f-967f4d623cf2 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.042400 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6336b-5766-4cb3-9c6f-967f4d623cf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:32.042702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6336b-5766-4cb3-9c6f-967f4d623cf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.044115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6336b-5766-4cb3-9c6f-967f4d623cf2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.051815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:32.056587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 553/1099] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:32.061969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128aa80-1648-414c-aa2a-3e2b9833c1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:32.062135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6336b-5766-4cb3-9c6f-967f4d623cf2 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:32.062651 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128aa80-1648-414c-aa2a-3e2b9833c1f4 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.063225 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128aa80-1648-414c-aa2a-3e2b9833c1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.063498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128aa80-1648-414c-aa2a-3e2b9833c1f4 None admin] PATH_INFO: `/v3/users/1937dfec03b74d76a29ee52e482119e9` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.064644 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128aa80-1648-414c-aa2a-3e2b9833c1f4 None admin] RBAC: Authorizing `identity:get_user(user_id=1937dfec03b74d76a29ee52e482119e9)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.066067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128aa80-1648-414c-aa2a-3e2b9833c1f4 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.067758 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 547/1100] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/users/1937dfec03b74d76a29ee52e482119e9 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:32.073691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128aa80-1648-414c-aa2a-3e2b9833c1f4 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:32.160953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:32.165627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6336b-5766-4cb3-9c6f-967f4d623cf2 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:32.174534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128aa80-1648-414c-aa2a-3e2b9833c1f4 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:32.200208 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef0531-b48f-4140-a5e0-e327788cc6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:32.200839 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef0531-b48f-4140-a5e0-e327788cc6fc None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.201118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef0531-b48f-4140-a5e0-e327788cc6fc None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.201388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef0531-b48f-4140-a5e0-e327788cc6fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.207971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef0531-b48f-4140-a5e0-e327788cc6fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.208180 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b8826-7817-4f39-ba69-f5f72f62a166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:32.208774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b8826-7817-4f39-ba69-f5f72f62a166 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.209047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b8826-7817-4f39-ba69-f5f72f62a166 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.209311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b8826-7817-4f39-ba69-f5f72f62a166 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.209404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef0531-b48f-4140-a5e0-e327788cc6fc None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-9eef0531-b48f-4140-a5e0-e327788cc6fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:32.221980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:32.222934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:32.223876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:32.223876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:32.223876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:32.223876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:32.223876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:32.223876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:32.223876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:32.223876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:32.223876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 554/1101] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:32.229758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef0531-b48f-4140-a5e0-e327788cc6fc None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:32.277842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:32.284715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b8826-7817-4f39-ba69-f5f72f62a166 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.286181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b8826-7817-4f39-ba69-f5f72f62a166 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.330424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef0531-b48f-4140-a5e0-e327788cc6fc None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:32.365166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917af42e-82ce-407b-b8a1-f61f204adc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:32.365766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917af42e-82ce-407b-b8a1-f61f204adc34 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.366036 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917af42e-82ce-407b-b8a1-f61f204adc34 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.366301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917af42e-82ce-407b-b8a1-f61f204adc34 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.367170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917af42e-82ce-407b-b8a1-f61f204adc34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:32.367397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917af42e-82ce-407b-b8a1-f61f204adc34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.368802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917af42e-82ce-407b-b8a1-f61f204adc34 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.381828 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 555/1102] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:36:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:32.391768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 548/1103] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:36:32.391940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917af42e-82ce-407b-b8a1-f61f204adc34 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:32.392776 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b8826-7817-4f39-ba69-f5f72f62a166 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:32.407649 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:32.486394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:32.489815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917af42e-82ce-407b-b8a1-f61f204adc34 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:32.492961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b8826-7817-4f39-ba69-f5f72f62a166 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:32.526763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1b947-b79d-4f36-93b9-b1739db2fe16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:32.527411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1b947-b79d-4f36-93b9-b1739db2fe16 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.527690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1b947-b79d-4f36-93b9-b1739db2fe16 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.527959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1b947-b79d-4f36-93b9-b1739db2fe16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.528685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1b947-b79d-4f36-93b9-b1739db2fe16 None admin] RBAC: Adding query filter params (role.id=5bb2daf5cef542028e8802f2f06a46bd, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=ef72bb95992c4a21ae2dbb2a08c51421) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:32.528992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1b947-b79d-4f36-93b9-b1739db2fe16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.529114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08403f5-f922-4ce5-91a2-41a07fd8d8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:32.529742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08403f5-f922-4ce5-91a2-41a07fd8d8a4 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.530021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08403f5-f922-4ce5-91a2-41a07fd8d8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.530292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08403f5-f922-4ce5-91a2-41a07fd8d8a4 None admin] PATH_INFO: `/v3/projects/140b555fb3954bd98f1c35f4297ffbc0` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.530423 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1b947-b79d-4f36-93b9-b1739db2fe16 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.531442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08403f5-f922-4ce5-91a2-41a07fd8d8a4 None admin] RBAC: Authorizing `identity:get_project(project_id=140b555fb3954bd98f1c35f4297ffbc0)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.532863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08403f5-f922-4ce5-91a2-41a07fd8d8a4 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.534491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 549/1104] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:32 2022] GET /identity/v3/projects/140b555fb3954bd98f1c35f4297ffbc0 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:32.540000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 556/1105] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:32 2022] GET /identity/v3/role_assignments?user.id=ef72bb95992c4a21ae2dbb2a08c51421&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=5bb2daf5cef542028e8802f2f06a46bd => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:32.542176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08403f5-f922-4ce5-91a2-41a07fd8d8a4 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:32.651573 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08403f5-f922-4ce5-91a2-41a07fd8d8a4 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:32.688069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23771179-8d44-4157-a23b-a53d86d1bdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:32.688674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23771179-8d44-4157-a23b-a53d86d1bdb9 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.688950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23771179-8d44-4157-a23b-a53d86d1bdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.689217 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23771179-8d44-4157-a23b-a53d86d1bdb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.689938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23771179-8d44-4157-a23b-a53d86d1bdb9 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=140b555fb3954bd98f1c35f4297ffbc0, user.id=1937dfec03b74d76a29ee52e482119e9) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:32.690243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23771179-8d44-4157-a23b-a53d86d1bdb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.691708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23771179-8d44-4157-a23b-a53d86d1bdb9 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.697927 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 550/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:32 2022] GET /identity/v3/role_assignments?user.id=1937dfec03b74d76a29ee52e482119e9&scope.project.id=140b555fb3954bd98f1c35f4297ffbc0&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:34.686287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2097d-afe2-4da4-a561-2c38f5f6b138 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:34.686846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2097d-afe2-4da4-a561-2c38f5f6b138 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:34.686846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2097d-afe2-4da4-a561-2c38f5f6b138 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:34.688218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 557/1107] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:34.693339 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35069588-be49-4d5e-8509-93cb7590ad94 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:34.693787 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35069588-be49-4d5e-8509-93cb7590ad94 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:34.693881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35069588-be49-4d5e-8509-93cb7590ad94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:34.712387 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-35069588-be49-4d5e-8509-93cb7590ad94 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:34.713579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-35069588-be49-4d5e-8509-93cb7590ad94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:34.736265 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:34.780796 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 551/1108] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:35.058148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2097d-afe2-4da4-a561-2c38f5f6b138 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:35.141677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:35.161065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2097d-afe2-4da4-a561-2c38f5f6b138 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:35.194141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb562d5-4c28-498c-b0ad-19c3602a75b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:35.194759 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb562d5-4c28-498c-b0ad-19c3602a75b9 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:35.195032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb562d5-4c28-498c-b0ad-19c3602a75b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:35.196012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb562d5-4c28-498c-b0ad-19c3602a75b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:35.197125 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb562d5-4c28-498c-b0ad-19c3602a75b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:35.198543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb562d5-4c28-498c-b0ad-19c3602a75b9 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:35.200308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 558/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:35.206150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35069588-be49-4d5e-8509-93cb7590ad94 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:35.306059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35069588-be49-4d5e-8509-93cb7590ad94 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:35.339503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e84ebd-819c-4687-8297-95338075a4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:35.340034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e84ebd-819c-4687-8297-95338075a4fd None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:35.340307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e84ebd-819c-4687-8297-95338075a4fd None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:35.340572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e84ebd-819c-4687-8297-95338075a4fd None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:35.341222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e84ebd-819c-4687-8297-95338075a4fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:35.342636 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e84ebd-819c-4687-8297-95338075a4fd None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:35.363601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 552/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:36:35 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:37.292263 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e51cab-1c3a-4ce5-8104-7021a21a0c3e None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:37.292808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e51cab-1c3a-4ce5-8104-7021a21a0c3e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:37.292808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e51cab-1c3a-4ce5-8104-7021a21a0c3e None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:37.294197 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 559/1111] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:37.300351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cebafa-db0b-4750-8f92-3d292de22f31 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:37.300873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cebafa-db0b-4750-8f92-3d292de22f31 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:37.300873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cebafa-db0b-4750-8f92-3d292de22f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:37.320655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-c8cebafa-db0b-4750-8f92-3d292de22f31 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:37.321934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c8cebafa-db0b-4750-8f92-3d292de22f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:37.332675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:37.395700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 553/1112] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:37.673228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e51cab-1c3a-4ce5-8104-7021a21a0c3e None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:37.772872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e51cab-1c3a-4ce5-8104-7021a21a0c3e None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:37.807762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be551bd-1cab-4484-9222-f55bba440de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:37.808389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be551bd-1cab-4484-9222-f55bba440de1 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:37.808669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be551bd-1cab-4484-9222-f55bba440de1 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:37.808943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be551bd-1cab-4484-9222-f55bba440de1 None admin] PATH_INFO: `/v3/roles/83a1e8de33194004a2a672104465c819` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:37.810335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be551bd-1cab-4484-9222-f55bba440de1 None admin] RBAC: Authorizing `identity:get_role(role_id=83a1e8de33194004a2a672104465c819)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:37.811738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be551bd-1cab-4484-9222-f55bba440de1 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:37.813073 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 560/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:37 2022] GET /identity/v3/roles/83a1e8de33194004a2a672104465c819 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:37.819044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cebafa-db0b-4750-8f92-3d292de22f31 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:37.919691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cebafa-db0b-4750-8f92-3d292de22f31 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:37.953134 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25200068-f661-4b7e-a3f6-34b7f5902521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:37.953687 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25200068-f661-4b7e-a3f6-34b7f5902521 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:37.953961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25200068-f661-4b7e-a3f6-34b7f5902521 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:37.954228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25200068-f661-4b7e-a3f6-34b7f5902521 None admin] PATH_INFO: `/v3/users/d3a62c8d19d4410e95c7ba325db7f31b` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:37.967890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25200068-f661-4b7e-a3f6-34b7f5902521 None admin] RBAC: Authorizing `identity:get_user(user_id=d3a62c8d19d4410e95c7ba325db7f31b)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:37.969336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25200068-f661-4b7e-a3f6-34b7f5902521 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:37.971072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 554/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:37 2022] GET /identity/v3/users/d3a62c8d19d4410e95c7ba325db7f31b => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:37.977907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be551bd-1cab-4484-9222-f55bba440de1 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:37.997219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:38.077289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be551bd-1cab-4484-9222-f55bba440de1 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:38.111451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318aac1-b344-459e-b903-0424e028c5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:38.112051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318aac1-b344-459e-b903-0424e028c5b6 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:38.112329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318aac1-b344-459e-b903-0424e028c5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:38.112602 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318aac1-b344-459e-b903-0424e028c5b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:38.178375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318aac1-b344-459e-b903-0424e028c5b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:38.180668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318aac1-b344-459e-b903-0424e028c5b6 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:38.199423 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:38.282293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 561/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:36:38.288265 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25200068-f661-4b7e-a3f6-34b7f5902521 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:38.291800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:38.392658 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25200068-f661-4b7e-a3f6-34b7f5902521 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:38.425874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad0948-9593-4a5d-93b0-a01f734c7e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:38.426483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad0948-9593-4a5d-93b0-a01f734c7e28 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:38.426760 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad0948-9593-4a5d-93b0-a01f734c7e28 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:38.427031 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad0948-9593-4a5d-93b0-a01f734c7e28 None admin] PATH_INFO: `/v3/projects/140b555fb3954bd98f1c35f4297ffbc0` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:38.428169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad0948-9593-4a5d-93b0-a01f734c7e28 None admin] RBAC: Authorizing `identity:get_project(project_id=140b555fb3954bd98f1c35f4297ffbc0)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:38.430588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad0948-9593-4a5d-93b0-a01f734c7e28 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:38.432239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 555/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:38 2022] GET /identity/v3/projects/140b555fb3954bd98f1c35f4297ffbc0 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:38.438468 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318aac1-b344-459e-b903-0424e028c5b6 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:38.536054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318aac1-b344-459e-b903-0424e028c5b6 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:38.553279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:38.570722 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a9d64-8e3b-4b1e-a1b2-190f14191489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:38.571495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a9d64-8e3b-4b1e-a1b2-190f14191489 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:38.571775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a9d64-8e3b-4b1e-a1b2-190f14191489 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:38.572046 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a9d64-8e3b-4b1e-a1b2-190f14191489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:38.572761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a9d64-8e3b-4b1e-a1b2-190f14191489 None admin] RBAC: Adding query filter params (role.id=83a1e8de33194004a2a672104465c819, scope.project.id=140b555fb3954bd98f1c35f4297ffbc0, user.id=d3a62c8d19d4410e95c7ba325db7f31b) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:38.573067 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a9d64-8e3b-4b1e-a1b2-190f14191489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:38.574495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a9d64-8e3b-4b1e-a1b2-190f14191489 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:38.579892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 562/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:38 2022] GET /identity/v3/role_assignments?user.id=d3a62c8d19d4410e95c7ba325db7f31b&scope.project.id=140b555fb3954bd98f1c35f4297ffbc0&role.id=83a1e8de33194004a2a672104465c819 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:40.468137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6acdc-1ff7-459a-b49c-68ecc897b2a0 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:40.468673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6acdc-1ff7-459a-b49c-68ecc897b2a0 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:40.468673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6acdc-1ff7-459a-b49c-68ecc897b2a0 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:40.469981 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 556/1118] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:40.475353 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c195ada-3066-4d43-92b3-726de8343efe None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:40.475786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c195ada-3066-4d43-92b3-726de8343efe None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:40.475907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c195ada-3066-4d43-92b3-726de8343efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:40.495570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-7c195ada-3066-4d43-92b3-726de8343efe None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:40.496769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7c195ada-3066-4d43-92b3-726de8343efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:40.565350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 563/1119] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:40.843394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6acdc-1ff7-459a-b49c-68ecc897b2a0 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:40.914873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:40.948215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6acdc-1ff7-459a-b49c-68ecc897b2a0 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:40.986816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa439a8-2c98-4f4c-9adf-c1783a7d284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:40.987481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa439a8-2c98-4f4c-9adf-c1783a7d284d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:40.987759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa439a8-2c98-4f4c-9adf-c1783a7d284d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:40.988026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa439a8-2c98-4f4c-9adf-c1783a7d284d None admin] PATH_INFO: `/v3/roles/83a1e8de33194004a2a672104465c819` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:40.989450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa439a8-2c98-4f4c-9adf-c1783a7d284d None admin] RBAC: Authorizing `identity:get_role(role_id=83a1e8de33194004a2a672104465c819)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:40.990891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa439a8-2c98-4f4c-9adf-c1783a7d284d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:40.992850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 557/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:40 2022] GET /identity/v3/roles/83a1e8de33194004a2a672104465c819 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:40.998899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c195ada-3066-4d43-92b3-726de8343efe None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:41.103344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c195ada-3066-4d43-92b3-726de8343efe None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:41.116685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:41.138773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24050912-355c-4e3a-aff5-547fdb2c7a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:41.139688 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24050912-355c-4e3a-aff5-547fdb2c7a3f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:41.139966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24050912-355c-4e3a-aff5-547fdb2c7a3f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:41.140235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24050912-355c-4e3a-aff5-547fdb2c7a3f None admin] PATH_INFO: `/v3/users/d3a62c8d19d4410e95c7ba325db7f31b` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:41.141378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24050912-355c-4e3a-aff5-547fdb2c7a3f None admin] RBAC: Authorizing `identity:get_user(user_id=d3a62c8d19d4410e95c7ba325db7f31b)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:41.142801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24050912-355c-4e3a-aff5-547fdb2c7a3f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:41.144432 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 564/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:40 2022] GET /identity/v3/users/d3a62c8d19d4410e95c7ba325db7f31b => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:41.151184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa439a8-2c98-4f4c-9adf-c1783a7d284d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:41.251627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa439a8-2c98-4f4c-9adf-c1783a7d284d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:41.289408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbc1dd-2c8f-4c7a-9dda-817b4c96f434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:41.290023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbc1dd-2c8f-4c7a-9dda-817b4c96f434 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:41.290303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbc1dd-2c8f-4c7a-9dda-817b4c96f434 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:41.290574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbc1dd-2c8f-4c7a-9dda-817b4c96f434 None admin] PATH_INFO: `/v3/projects/140b555fb3954bd98f1c35f4297ffbc0` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:41.291726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbc1dd-2c8f-4c7a-9dda-817b4c96f434 None admin] RBAC: Authorizing `identity:get_project(project_id=140b555fb3954bd98f1c35f4297ffbc0)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:41.293163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbc1dd-2c8f-4c7a-9dda-817b4c96f434 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:41.294802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 558/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:41 2022] GET /identity/v3/projects/140b555fb3954bd98f1c35f4297ffbc0 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:41.300836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24050912-355c-4e3a-aff5-547fdb2c7a3f None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:41.400108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24050912-355c-4e3a-aff5-547fdb2c7a3f None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:41.434356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d93d2-4646-414b-b518-d8c138a11f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:41.435014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d93d2-4646-414b-b518-d8c138a11f74 None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:41.435324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d93d2-4646-414b-b518-d8c138a11f74 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:41.435596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d93d2-4646-414b-b518-d8c138a11f74 None admin] PATH_INFO: `/v3/projects/140b555fb3954bd98f1c35f4297ffbc0/users/d3a62c8d19d4410e95c7ba325db7f31b/roles/83a1e8de33194004a2a672104465c819` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:41.437735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d93d2-4646-414b-b518-d8c138a11f74 None admin] RBAC: Authorizing `identity:create_grant(project_id=140b555fb3954bd98f1c35f4297ffbc0, user_id=d3a62c8d19d4410e95c7ba325db7f31b, role_id=83a1e8de33194004a2a672104465c819)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:41.439185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d93d2-4646-414b-b518-d8c138a11f74 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:41.453055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 565/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:36:41 2022] PUT /identity/v3/projects/140b555fb3954bd98f1c35f4297ffbc0/users/d3a62c8d19d4410e95c7ba325db7f31b/roles/83a1e8de33194004a2a672104465c819 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:36:43.344569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835bba2-18d0-4857-986d-56b5fc633c23 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:43.345091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835bba2-18d0-4857-986d-56b5fc633c23 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:43.345091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835bba2-18d0-4857-986d-56b5fc633c23 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:43.346343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 559/1124] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:43.352533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee5375-8511-482b-a420-cc4ef6598e1d None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:43.353007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee5375-8511-482b-a420-cc4ef6598e1d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:43.353107 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee5375-8511-482b-a420-cc4ef6598e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:43.371437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b0ee5375-8511-482b-a420-cc4ef6598e1d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:43.373124 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b0ee5375-8511-482b-a420-cc4ef6598e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:43.409841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:43.446824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 566/1125] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:43.724205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835bba2-18d0-4857-986d-56b5fc633c23 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:43.738787 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:43.847557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835bba2-18d0-4857-986d-56b5fc633c23 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:43.889322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e511e10-748c-4ab5-906b-e0016eac433d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:43.890179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e511e10-748c-4ab5-906b-e0016eac433d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:43.890457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e511e10-748c-4ab5-906b-e0016eac433d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:43.890846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e511e10-748c-4ab5-906b-e0016eac433d None admin] PATH_INFO: `/v3/roles/83a1e8de33194004a2a672104465c819` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:43.892590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e511e10-748c-4ab5-906b-e0016eac433d None admin] RBAC: Authorizing `identity:get_role(role_id=83a1e8de33194004a2a672104465c819)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:43.894528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e511e10-748c-4ab5-906b-e0016eac433d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:43.896177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 560/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:43 2022] GET /identity/v3/roles/83a1e8de33194004a2a672104465c819 => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:43.905674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee5375-8511-482b-a420-cc4ef6598e1d None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:44.033887 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee5375-8511-482b-a420-cc4ef6598e1d None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:44.074353 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a0e87-1cc6-4694-8f2a-2a26d25bfdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:44.075111 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a0e87-1cc6-4694-8f2a-2a26d25bfdb1 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:44.075449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a0e87-1cc6-4694-8f2a-2a26d25bfdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:44.075843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a0e87-1cc6-4694-8f2a-2a26d25bfdb1 None admin] PATH_INFO: `/v3/users/d3a62c8d19d4410e95c7ba325db7f31b` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:44.077182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a0e87-1cc6-4694-8f2a-2a26d25bfdb1 None admin] RBAC: Authorizing `identity:get_user(user_id=d3a62c8d19d4410e95c7ba325db7f31b)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:44.078898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a0e87-1cc6-4694-8f2a-2a26d25bfdb1 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:44.081021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 567/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:43 2022] GET /identity/v3/users/d3a62c8d19d4410e95c7ba325db7f31b => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:44.088329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e511e10-748c-4ab5-906b-e0016eac433d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:44.171695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:44.203019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e511e10-748c-4ab5-906b-e0016eac433d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:44.243585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83917e6c-a472-4290-8c30-11aebc95567e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:44.244188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83917e6c-a472-4290-8c30-11aebc95567e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:44.244461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83917e6c-a472-4290-8c30-11aebc95567e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:44.244728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83917e6c-a472-4290-8c30-11aebc95567e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:44.324003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83917e6c-a472-4290-8c30-11aebc95567e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:44.325730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83917e6c-a472-4290-8c30-11aebc95567e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:44.420472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:44.454520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 561/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:36:44.461593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a0e87-1cc6-4694-8f2a-2a26d25bfdb1 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:44.523947 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:44.603354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a0e87-1cc6-4694-8f2a-2a26d25bfdb1 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:44.645270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f6e02-5c3e-4c22-aa6b-f6d8e3c43ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:44.645912 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f6e02-5c3e-4c22-aa6b-f6d8e3c43ee3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:44.646188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f6e02-5c3e-4c22-aa6b-f6d8e3c43ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:44.646458 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f6e02-5c3e-4c22-aa6b-f6d8e3c43ee3 None admin] PATH_INFO: `/v3/projects/140b555fb3954bd98f1c35f4297ffbc0` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:44.647746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f6e02-5c3e-4c22-aa6b-f6d8e3c43ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=140b555fb3954bd98f1c35f4297ffbc0)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:44.650582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f6e02-5c3e-4c22-aa6b-f6d8e3c43ee3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:44.652595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 568/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:44 2022] GET /identity/v3/projects/140b555fb3954bd98f1c35f4297ffbc0 => generated 311 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:44.659147 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83917e6c-a472-4290-8c30-11aebc95567e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:44.782096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83917e6c-a472-4290-8c30-11aebc95567e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:44.817868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0b486-d056-47cd-9299-5c80b6d8d424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:44.818475 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0b486-d056-47cd-9299-5c80b6d8d424 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:44.818758 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0b486-d056-47cd-9299-5c80b6d8d424 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:44.819030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0b486-d056-47cd-9299-5c80b6d8d424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:44.820331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0b486-d056-47cd-9299-5c80b6d8d424 None admin] RBAC: Adding query filter params (role.id=83a1e8de33194004a2a672104465c819, scope.project.id=140b555fb3954bd98f1c35f4297ffbc0, user.id=d3a62c8d19d4410e95c7ba325db7f31b) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:44.820700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0b486-d056-47cd-9299-5c80b6d8d424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:44.822138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0b486-d056-47cd-9299-5c80b6d8d424 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:44.828284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 562/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:44 2022] GET /identity/v3/role_assignments?user.id=d3a62c8d19d4410e95c7ba325db7f31b&scope.project.id=140b555fb3954bd98f1c35f4297ffbc0&role.id=83a1e8de33194004a2a672104465c819 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:46.941507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596fd7f-0225-4ceb-8617-89aead9b96e9 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:46.942066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596fd7f-0225-4ceb-8617-89aead9b96e9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:46.942066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596fd7f-0225-4ceb-8617-89aead9b96e9 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:46.943579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 569/1131] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:46.949289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db5fe6-10d1-469e-82cd-31ea01da565e None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:46.949708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db5fe6-10d1-469e-82cd-31ea01da565e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:46.949825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db5fe6-10d1-469e-82cd-31ea01da565e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:46.967843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b4db5fe6-10d1-469e-82cd-31ea01da565e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:46.969050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b4db5fe6-10d1-469e-82cd-31ea01da565e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:47.001766 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:47.044155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 563/1132] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:47.318105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596fd7f-0225-4ceb-8617-89aead9b96e9 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:47.422095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596fd7f-0225-4ceb-8617-89aead9b96e9 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:47.430146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:47.457755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee944d4-a752-46ae-b2e1-31b93edaa5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:47.458377 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee944d4-a752-46ae-b2e1-31b93edaa5e6 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:47.458651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee944d4-a752-46ae-b2e1-31b93edaa5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:47.458916 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee944d4-a752-46ae-b2e1-31b93edaa5e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:47.460055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee944d4-a752-46ae-b2e1-31b93edaa5e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:47.462315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee944d4-a752-46ae-b2e1-31b93edaa5e6 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:47.464027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 570/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:47.469862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db5fe6-10d1-469e-82cd-31ea01da565e None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:47.566466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db5fe6-10d1-469e-82cd-31ea01da565e None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:47.600599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f547d2f-a8bd-4005-a297-061ed668c054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:47.601223 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f547d2f-a8bd-4005-a297-061ed668c054 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:47.601495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f547d2f-a8bd-4005-a297-061ed668c054 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:47.601760 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f547d2f-a8bd-4005-a297-061ed668c054 None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:47.602415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f547d2f-a8bd-4005-a297-061ed668c054 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:47.603861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f547d2f-a8bd-4005-a297-061ed668c054 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:47.630354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 564/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:36:47 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:49.734820 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f730c-1dda-48ff-8ad9-bf3c0575b6d9 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:49.735498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f730c-1dda-48ff-8ad9-bf3c0575b6d9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:49.735498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f730c-1dda-48ff-8ad9-bf3c0575b6d9 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:49.736751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 571/1135] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:49.742831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e345a8a-f94f-4cf4-b2fd-a2f5ca42d74f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:49.743347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e345a8a-f94f-4cf4-b2fd-a2f5ca42d74f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:49.743452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e345a8a-f94f-4cf4-b2fd-a2f5ca42d74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:49.762412 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-6e345a8a-f94f-4cf4-b2fd-a2f5ca42d74f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:49.763918 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6e345a8a-f94f-4cf4-b2fd-a2f5ca42d74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:49.780043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:49.838671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 565/1136] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:50.109071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f730c-1dda-48ff-8ad9-bf3c0575b6d9 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:50.220392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f730c-1dda-48ff-8ad9-bf3c0575b6d9 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:50.259327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786de1af-3308-4bc7-aa25-9b3b69390c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:50.259984 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786de1af-3308-4bc7-aa25-9b3b69390c15 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:50.260286 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786de1af-3308-4bc7-aa25-9b3b69390c15 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:50.260557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786de1af-3308-4bc7-aa25-9b3b69390c15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:50.261763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786de1af-3308-4bc7-aa25-9b3b69390c15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:50.263452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786de1af-3308-4bc7-aa25-9b3b69390c15 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:50.265626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 572/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:50.273177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e345a8a-f94f-4cf4-b2fd-a2f5ca42d74f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:50.397530 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e345a8a-f94f-4cf4-b2fd-a2f5ca42d74f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:50.435058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed092ffb-9995-4e81-979e-bc22033630cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:50.435910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed092ffb-9995-4e81-979e-bc22033630cf None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:50.436375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed092ffb-9995-4e81-979e-bc22033630cf None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:50.436893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed092ffb-9995-4e81-979e-bc22033630cf None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:50.437817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed092ffb-9995-4e81-979e-bc22033630cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:50.441520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed092ffb-9995-4e81-979e-bc22033630cf None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:50.468571 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 566/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:36:50 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:52.478452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d9a33-deea-42dc-9949-4d34457909cb None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:52.479010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d9a33-deea-42dc-9949-4d34457909cb None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:52.479010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d9a33-deea-42dc-9949-4d34457909cb None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:52.480368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 573/1139] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:52.485513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0378d1-a9f9-4dd7-be48-27c58d9c517d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:52.485961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0378d1-a9f9-4dd7-be48-27c58d9c517d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:52.486063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0378d1-a9f9-4dd7-be48-27c58d9c517d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:52.499315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:52.505330 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-0b0378d1-a9f9-4dd7-be48-27c58d9c517d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:52.506783 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0b0378d1-a9f9-4dd7-be48-27c58d9c517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:52.576319 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 567/1140] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:52.845018 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d9a33-deea-42dc-9949-4d34457909cb None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:52.894945 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:52.969632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d9a33-deea-42dc-9949-4d34457909cb None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:53.003611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d08138-5352-4ef8-8f49-b6badfa55735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:53.004236 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d08138-5352-4ef8-8f49-b6badfa55735 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:53.004517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d08138-5352-4ef8-8f49-b6badfa55735 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:53.004789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d08138-5352-4ef8-8f49-b6badfa55735 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:53.015773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d08138-5352-4ef8-8f49-b6badfa55735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-06d08138-5352-4ef8-8f49-b6badfa55735 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:53.020550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:53.021729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:53.024058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 574/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:53.028445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0378d1-a9f9-4dd7-be48-27c58d9c517d None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:53.129134 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0378d1-a9f9-4dd7-be48-27c58d9c517d None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:53.168280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f71d04-af61-4257-85e3-2e77c2390a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:53.168876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f71d04-af61-4257-85e3-2e77c2390a16 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:53.169156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f71d04-af61-4257-85e3-2e77c2390a16 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:53.169423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f71d04-af61-4257-85e3-2e77c2390a16 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:53.170091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f71d04-af61-4257-85e3-2e77c2390a16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:53.170370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f71d04-af61-4257-85e3-2e77c2390a16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:53.171759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f71d04-af61-4257-85e3-2e77c2390a16 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:53.173849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:53.181088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 568/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:53.186847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d08138-5352-4ef8-8f49-b6badfa55735 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:53.285201 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d08138-5352-4ef8-8f49-b6badfa55735 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:53.286729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:53.319881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b0263-7834-4ff4-baa8-2af90bf50b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:53.320507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b0263-7834-4ff4-baa8-2af90bf50b09 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:53.320788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b0263-7834-4ff4-baa8-2af90bf50b09 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:53.321062 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b0263-7834-4ff4-baa8-2af90bf50b09 None admin] PATH_INFO: `/v3/users/7f9b2d2014944af999938cee80832efd` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:53.334466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b0263-7834-4ff4-baa8-2af90bf50b09 None admin] RBAC: Authorizing `identity:get_user(user_id=7f9b2d2014944af999938cee80832efd)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:53.336461 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b0263-7834-4ff4-baa8-2af90bf50b09 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:53.338103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 575/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:53 2022] GET /identity/v3/users/7f9b2d2014944af999938cee80832efd => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:53.344035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f71d04-af61-4257-85e3-2e77c2390a16 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:53.449027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f71d04-af61-4257-85e3-2e77c2390a16 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:53.488459 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c816400-e817-48ab-9643-4534b8deca14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:53.489113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c816400-e817-48ab-9643-4534b8deca14 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:53.489318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c816400-e817-48ab-9643-4534b8deca14 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:53.489566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c816400-e817-48ab-9643-4534b8deca14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:53.553707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:53.557626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c816400-e817-48ab-9643-4534b8deca14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:53.559192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c816400-e817-48ab-9643-4534b8deca14 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:53.657408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 569/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:36:53.663534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b0263-7834-4ff4-baa8-2af90bf50b09 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:53.761791 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b0263-7834-4ff4-baa8-2af90bf50b09 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:53.795043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26492ee5-b693-4e69-9f0e-7ce51f0019fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:53.795696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26492ee5-b693-4e69-9f0e-7ce51f0019fa None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:53.795970 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26492ee5-b693-4e69-9f0e-7ce51f0019fa None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:53.796235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26492ee5-b693-4e69-9f0e-7ce51f0019fa None admin] PATH_INFO: `/v3/projects/a1d4530354104485a40c40715ec3f884` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:53.797332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26492ee5-b693-4e69-9f0e-7ce51f0019fa None admin] RBAC: Authorizing `identity:get_project(project_id=a1d4530354104485a40c40715ec3f884)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:53.798797 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26492ee5-b693-4e69-9f0e-7ce51f0019fa None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:53.800466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 576/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:53 2022] GET /identity/v3/projects/a1d4530354104485a40c40715ec3f884 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:53.806550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c816400-e817-48ab-9643-4534b8deca14 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:53.911841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:53.916852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c816400-e817-48ab-9643-4534b8deca14 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:53.955185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b58d8-827e-45b3-af03-cdf0c48a9611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:53.955804 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b58d8-827e-45b3-af03-cdf0c48a9611 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:53.956066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b58d8-827e-45b3-af03-cdf0c48a9611 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:53.956335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b58d8-827e-45b3-af03-cdf0c48a9611 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:53.957056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b58d8-827e-45b3-af03-cdf0c48a9611 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=a1d4530354104485a40c40715ec3f884, user.id=7f9b2d2014944af999938cee80832efd) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:53.957383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b58d8-827e-45b3-af03-cdf0c48a9611 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:53.958815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b58d8-827e-45b3-af03-cdf0c48a9611 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:53.964476 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 570/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:53 2022] GET /identity/v3/role_assignments?user.id=7f9b2d2014944af999938cee80832efd&scope.project.id=a1d4530354104485a40c40715ec3f884&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:55.921030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48e512-8499-4706-ad1a-4a735ff6f364 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:55.921593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48e512-8499-4706-ad1a-4a735ff6f364 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:55.921593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48e512-8499-4706-ad1a-4a735ff6f364 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:55.922923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 577/1147] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:55.928079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c757b-0c32-4f77-af4a-be20a9a4d101 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:55.928536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c757b-0c32-4f77-af4a-be20a9a4d101 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:55.928632 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c757b-0c32-4f77-af4a-be20a9a4d101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:55.946968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-308c757b-0c32-4f77-af4a-be20a9a4d101 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:55.948282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-308c757b-0c32-4f77-af4a-be20a9a4d101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:56.014714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 571/1148] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:56.285855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48e512-8499-4706-ad1a-4a735ff6f364 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:56.314362 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:56.386052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48e512-8499-4706-ad1a-4a735ff6f364 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:56.418183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902729a2-6a09-49f0-b2dd-3d33d0af3273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:56.418801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902729a2-6a09-49f0-b2dd-3d33d0af3273 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:56.419074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902729a2-6a09-49f0-b2dd-3d33d0af3273 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:56.419366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902729a2-6a09-49f0-b2dd-3d33d0af3273 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:56.429595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902729a2-6a09-49f0-b2dd-3d33d0af3273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-902729a2-6a09-49f0-b2dd-3d33d0af3273 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:56.434266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:56.435266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:36:56.436143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 578/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:56.441386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c757b-0c32-4f77-af4a-be20a9a4d101 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:56.531887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:56.541263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c757b-0c32-4f77-af4a-be20a9a4d101 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:56.574350 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e443c-8d3a-422d-a1d5-0797c607ca69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:56.575073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e443c-8d3a-422d-a1d5-0797c607ca69 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:56.575389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e443c-8d3a-422d-a1d5-0797c607ca69 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:56.575679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e443c-8d3a-422d-a1d5-0797c607ca69 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:56.576338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e443c-8d3a-422d-a1d5-0797c607ca69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:56.576650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e443c-8d3a-422d-a1d5-0797c607ca69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:56.578211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e443c-8d3a-422d-a1d5-0797c607ca69 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:56.587941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 572/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:56.594145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902729a2-6a09-49f0-b2dd-3d33d0af3273 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:56.685168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:56.697674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902729a2-6a09-49f0-b2dd-3d33d0af3273 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:56.731216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df79b5e-ef3c-4632-807e-b0d9d7ac3991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:56.731848 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df79b5e-ef3c-4632-807e-b0d9d7ac3991 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:56.732128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df79b5e-ef3c-4632-807e-b0d9d7ac3991 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:56.732401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df79b5e-ef3c-4632-807e-b0d9d7ac3991 None admin] PATH_INFO: `/v3/users/7f9b2d2014944af999938cee80832efd` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:56.733548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df79b5e-ef3c-4632-807e-b0d9d7ac3991 None admin] RBAC: Authorizing `identity:get_user(user_id=7f9b2d2014944af999938cee80832efd)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:56.734979 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df79b5e-ef3c-4632-807e-b0d9d7ac3991 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:56.736603 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 579/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:56 2022] GET /identity/v3/users/7f9b2d2014944af999938cee80832efd => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:56.742517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e443c-8d3a-422d-a1d5-0797c607ca69 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:56.841290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e443c-8d3a-422d-a1d5-0797c607ca69 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:56.873531 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9973d47-73ac-4f4f-b935-89643f3990b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:56.874146 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9973d47-73ac-4f4f-b935-89643f3990b0 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:56.874417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9973d47-73ac-4f4f-b935-89643f3990b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:56.874683 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9973d47-73ac-4f4f-b935-89643f3990b0 None admin] PATH_INFO: `/v3/projects/a1d4530354104485a40c40715ec3f884` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:56.875751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9973d47-73ac-4f4f-b935-89643f3990b0 None admin] RBAC: Authorizing `identity:get_project(project_id=a1d4530354104485a40c40715ec3f884)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:56.877168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9973d47-73ac-4f4f-b935-89643f3990b0 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:56.878769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 573/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:56 2022] GET /identity/v3/projects/a1d4530354104485a40c40715ec3f884 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:56.885504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df79b5e-ef3c-4632-807e-b0d9d7ac3991 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:56.983981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df79b5e-ef3c-4632-807e-b0d9d7ac3991 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:57.018406 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd99f7-50d4-41bd-a7ca-8060f4e684d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:57.019064 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd99f7-50d4-41bd-a7ca-8060f4e684d8 None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:57.019375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd99f7-50d4-41bd-a7ca-8060f4e684d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:57.019650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd99f7-50d4-41bd-a7ca-8060f4e684d8 None admin] PATH_INFO: `/v3/projects/a1d4530354104485a40c40715ec3f884/users/7f9b2d2014944af999938cee80832efd/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:57.021566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd99f7-50d4-41bd-a7ca-8060f4e684d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1d4530354104485a40c40715ec3f884, user_id=7f9b2d2014944af999938cee80832efd, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:57.023029 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd99f7-50d4-41bd-a7ca-8060f4e684d8 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:57.035821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 580/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:36:56 2022] PUT /identity/v3/projects/a1d4530354104485a40c40715ec3f884/users/7f9b2d2014944af999938cee80832efd/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:36:58.964552 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65f019-cf8a-4019-9626-91ac72da55a7 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:58.965092 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65f019-cf8a-4019-9626-91ac72da55a7 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:58.965092 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65f019-cf8a-4019-9626-91ac72da55a7 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:58.966457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 574/1154] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:58.971860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49c857-b34f-4243-95a4-dd49fe4f1768 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:58.972329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49c857-b34f-4243-95a4-dd49fe4f1768 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:58.972421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49c857-b34f-4243-95a4-dd49fe4f1768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:58.991108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-9f49c857-b34f-4243-95a4-dd49fe4f1768 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:58.992899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9f49c857-b34f-4243-95a4-dd49fe4f1768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:59.003775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:59.061838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 581/1155] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:59.333694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65f019-cf8a-4019-9626-91ac72da55a7 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:59.358003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:59.436022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65f019-cf8a-4019-9626-91ac72da55a7 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:59.470016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ff7b7-1686-49b5-bced-8c7d072cc7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:59.470634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ff7b7-1686-49b5-bced-8c7d072cc7e8 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:59.470912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ff7b7-1686-49b5-bced-8c7d072cc7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:59.471210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ff7b7-1686-49b5-bced-8c7d072cc7e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:59.481837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ff7b7-1686-49b5-bced-8c7d072cc7e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-9d9ff7b7-1686-49b5-bced-8c7d072cc7e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:59.486562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:59.487676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:36:59.488543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 575/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:59.494047 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49c857-b34f-4243-95a4-dd49fe4f1768 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:59.593538 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49c857-b34f-4243-95a4-dd49fe4f1768 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:59.627049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000984f5-86e1-4dd4-947e-25d282d8a7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:59.627689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000984f5-86e1-4dd4-947e-25d282d8a7b2 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:59.627971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000984f5-86e1-4dd4-947e-25d282d8a7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:59.628242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000984f5-86e1-4dd4-947e-25d282d8a7b2 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:59.628904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000984f5-86e1-4dd4-947e-25d282d8a7b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:59.629207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000984f5-86e1-4dd4-947e-25d282d8a7b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:59.630568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000984f5-86e1-4dd4-947e-25d282d8a7b2 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:59.639437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 582/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:59.645289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ff7b7-1686-49b5-bced-8c7d072cc7e8 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:59.727927 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:59.746190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ff7b7-1686-49b5-bced-8c7d072cc7e8 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:59.782242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2f79c-92b0-46fb-8584-16c33086f116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:59.782888 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2f79c-92b0-46fb-8584-16c33086f116 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:59.783184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2f79c-92b0-46fb-8584-16c33086f116 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:59.783453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2f79c-92b0-46fb-8584-16c33086f116 None admin] PATH_INFO: `/v3/users/7f9b2d2014944af999938cee80832efd` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:59.784645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2f79c-92b0-46fb-8584-16c33086f116 None admin] RBAC: Authorizing `identity:get_user(user_id=7f9b2d2014944af999938cee80832efd)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:59.786113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2f79c-92b0-46fb-8584-16c33086f116 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:59.787802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 576/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:59 2022] GET /identity/v3/users/7f9b2d2014944af999938cee80832efd => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:59.794813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000984f5-86e1-4dd4-947e-25d282d8a7b2 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:59.808412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:59.900835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000984f5-86e1-4dd4-947e-25d282d8a7b2 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:59.935663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa4b2a-0d77-4e29-8de5-671146122d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:59.936256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa4b2a-0d77-4e29-8de5-671146122d10 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:59.936531 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa4b2a-0d77-4e29-8de5-671146122d10 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:59.936798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa4b2a-0d77-4e29-8de5-671146122d10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:00.002831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa4b2a-0d77-4e29-8de5-671146122d10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:00.004304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa4b2a-0d77-4e29-8de5-671146122d10 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:00.014160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:00.103098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 583/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:37:00.109116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2f79c-92b0-46fb-8584-16c33086f116 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:00.212921 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2f79c-92b0-46fb-8584-16c33086f116 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:00.247273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60a504-7345-401d-a7b4-f55f094f3beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:00.247889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60a504-7345-401d-a7b4-f55f094f3beb None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:00.248159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60a504-7345-401d-a7b4-f55f094f3beb None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:00.248425 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60a504-7345-401d-a7b4-f55f094f3beb None admin] PATH_INFO: `/v3/projects/a1d4530354104485a40c40715ec3f884` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:00.249518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60a504-7345-401d-a7b4-f55f094f3beb None admin] RBAC: Authorizing `identity:get_project(project_id=a1d4530354104485a40c40715ec3f884)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:00.250933 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60a504-7345-401d-a7b4-f55f094f3beb None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:00.252574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 577/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:37:00 2022] GET /identity/v3/projects/a1d4530354104485a40c40715ec3f884 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:00.259599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa4b2a-0d77-4e29-8de5-671146122d10 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:00.358612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa4b2a-0d77-4e29-8de5-671146122d10 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:00.368125 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:00.394023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b46199-074e-45e3-b4a5-17db124c35c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:00.394621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b46199-074e-45e3-b4a5-17db124c35c3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:00.394898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b46199-074e-45e3-b4a5-17db124c35c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:00.395193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b46199-074e-45e3-b4a5-17db124c35c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:00.395914 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b46199-074e-45e3-b4a5-17db124c35c3 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=a1d4530354104485a40c40715ec3f884, user.id=7f9b2d2014944af999938cee80832efd) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:00.396220 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b46199-074e-45e3-b4a5-17db124c35c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:00.398246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b46199-074e-45e3-b4a5-17db124c35c3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:00.403986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 584/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:37:00 2022] GET /identity/v3/role_assignments?user.id=7f9b2d2014944af999938cee80832efd&scope.project.id=a1d4530354104485a40c40715ec3f884&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:02.375065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cc30f-47e4-4f93-992d-0a760440f6ea None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:02.375746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cc30f-47e4-4f93-992d-0a760440f6ea None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:02.375746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cc30f-47e4-4f93-992d-0a760440f6ea None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:02.376858 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 578/1162] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:02.382032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485a028-f9a1-478e-bc85-a5c2920a4af1 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:02.382448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485a028-f9a1-478e-bc85-a5c2920a4af1 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:02.382581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485a028-f9a1-478e-bc85-a5c2920a4af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:02.401190 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e485a028-f9a1-478e-bc85-a5c2920a4af1 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:02.402390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e485a028-f9a1-478e-bc85-a5c2920a4af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:02.468516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 585/1163] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:02.738841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cc30f-47e4-4f93-992d-0a760440f6ea None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:02.758410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:02.840875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cc30f-47e4-4f93-992d-0a760440f6ea None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:02.874648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459138d6-19eb-4f06-86aa-eeea458ce411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:02.875272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459138d6-19eb-4f06-86aa-eeea458ce411 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:02.875548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459138d6-19eb-4f06-86aa-eeea458ce411 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:02.875814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459138d6-19eb-4f06-86aa-eeea458ce411 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:02.945373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459138d6-19eb-4f06-86aa-eeea458ce411 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:02.946861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459138d6-19eb-4f06-86aa-eeea458ce411 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:02.966593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:03.048850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 579/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:37:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:37:03.055303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485a028-f9a1-478e-bc85-a5c2920a4af1 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:03.153379 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485a028-f9a1-478e-bc85-a5c2920a4af1 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:03.154893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:03.188065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991dfc9-c44a-44ad-8d41-771b827638f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:03.188688 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991dfc9-c44a-44ad-8d41-771b827638f7 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:03.188970 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991dfc9-c44a-44ad-8d41-771b827638f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:03.189245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991dfc9-c44a-44ad-8d41-771b827638f7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:03.196371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f991dfc9-c44a-44ad-8d41-771b827638f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:03.197800 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f991dfc9-c44a-44ad-8d41-771b827638f7 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-f991dfc9-c44a-44ad-8d41-771b827638f7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:03.209653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:03.211023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:37:03.211911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 586/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 09:37:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:03.216806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459138d6-19eb-4f06-86aa-eeea458ce411 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:03.314381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459138d6-19eb-4f06-86aa-eeea458ce411 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:03.330922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:03.347558 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de979686-d2c2-4cc7-b0ae-f0c076510de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:03.348157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de979686-d2c2-4cc7-b0ae-f0c076510de1 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:03.348431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de979686-d2c2-4cc7-b0ae-f0c076510de1 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:03.348697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de979686-d2c2-4cc7-b0ae-f0c076510de1 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:03.349327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de979686-d2c2-4cc7-b0ae-f0c076510de1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:03.349620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de979686-d2c2-4cc7-b0ae-f0c076510de1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:03.351529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de979686-d2c2-4cc7-b0ae-f0c076510de1 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:03.358502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 580/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 09:37:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:05.262195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708a6b4-7b1d-433d-8fb8-28455dba4cc0 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:05.262772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708a6b4-7b1d-433d-8fb8-28455dba4cc0 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:05.262772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708a6b4-7b1d-433d-8fb8-28455dba4cc0 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:05.264366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 587/1167] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:05.269604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b10d6-ca41-4c0e-84ed-bcef7683ffe9 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:05.270015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b10d6-ca41-4c0e-84ed-bcef7683ffe9 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:05.270174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b10d6-ca41-4c0e-84ed-bcef7683ffe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:05.289084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-d74b10d6-ca41-4c0e-84ed-bcef7683ffe9 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:05.290269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d74b10d6-ca41-4c0e-84ed-bcef7683ffe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:05.357021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 581/1168] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:05.631643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708a6b4-7b1d-433d-8fb8-28455dba4cc0 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:05.734889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708a6b4-7b1d-433d-8fb8-28455dba4cc0 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:05.768299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d86ab9-edd0-4f74-b671-2c17a4bc7684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:05.768898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d86ab9-edd0-4f74-b671-2c17a4bc7684 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:05.769175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d86ab9-edd0-4f74-b671-2c17a4bc7684 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:05.769445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d86ab9-edd0-4f74-b671-2c17a4bc7684 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:05.770109 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d86ab9-edd0-4f74-b671-2c17a4bc7684 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:05.771494 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d86ab9-edd0-4f74-b671-2c17a4bc7684 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:05.787195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 588/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Wed Aug 31 09:37:05 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:37:07.732744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974ae17-592f-4a5d-b8a6-ce8aa6de0cb9 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:07.733355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974ae17-592f-4a5d-b8a6-ce8aa6de0cb9 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:07.733355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974ae17-592f-4a5d-b8a6-ce8aa6de0cb9 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:07.734657 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 582/1170] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:07.739985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323eb78a-3299-49db-9a0c-be9a03192067 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:07.740457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323eb78a-3299-49db-9a0c-be9a03192067 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:07.740570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323eb78a-3299-49db-9a0c-be9a03192067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:07.760378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-323eb78a-3299-49db-9a0c-be9a03192067 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:07.761589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-323eb78a-3299-49db-9a0c-be9a03192067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:07.778599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:07.830510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 589/1171] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:08.107835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974ae17-592f-4a5d-b8a6-ce8aa6de0cb9 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:08.210683 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974ae17-592f-4a5d-b8a6-ce8aa6de0cb9 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:08.223578 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:08.245086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e0f14-eda9-468f-87ae-7d726894ff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:08.245711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e0f14-eda9-468f-87ae-7d726894ff0f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:08.245985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e0f14-eda9-468f-87ae-7d726894ff0f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:08.246258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e0f14-eda9-468f-87ae-7d726894ff0f None admin] PATH_INFO: `/v3/domains/d9be7572b11441af8a9c6ad4ce88b31c` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:08.256495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e0f14-eda9-468f-87ae-7d726894ff0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=d9be7572b11441af8a9c6ad4ce88b31c)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:08.258569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e0f14-eda9-468f-87ae-7d726894ff0f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:08.260214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 583/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Aug 31 09:37:08 2022] GET /identity/v3/domains/d9be7572b11441af8a9c6ad4ce88b31c => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:08.266385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323eb78a-3299-49db-9a0c-be9a03192067 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:08.363718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323eb78a-3299-49db-9a0c-be9a03192067 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:08.398499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5877ab2-1037-4cd3-be47-4b00e439a8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:08.399100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877ab2-1037-4cd3-be47-4b00e439a8b5 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:08.399412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877ab2-1037-4cd3-be47-4b00e439a8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:08.399685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877ab2-1037-4cd3-be47-4b00e439a8b5 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:08.400336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5877ab2-1037-4cd3-be47-4b00e439a8b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:08.401769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5877ab2-1037-4cd3-be47-4b00e439a8b5 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:08.428043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 590/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:37:08 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:37:10.418632 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964816bf-69dd-4c38-a418-99f85e5c0453 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:10.419179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964816bf-69dd-4c38-a418-99f85e5c0453 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:10.419179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964816bf-69dd-4c38-a418-99f85e5c0453 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:10.420465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 584/1174] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:10.425777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddaf52-1b85-4a2d-8d1c-824d7e80285e None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:10.426230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddaf52-1b85-4a2d-8d1c-824d7e80285e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:10.426358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddaf52-1b85-4a2d-8d1c-824d7e80285e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:10.434111 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:10.446882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-55ddaf52-1b85-4a2d-8d1c-824d7e80285e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:10.448121 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-55ddaf52-1b85-4a2d-8d1c-824d7e80285e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:10.514337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 591/1175] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:10.782400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964816bf-69dd-4c38-a418-99f85e5c0453 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:10.883635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964816bf-69dd-4c38-a418-99f85e5c0453 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:10.917253 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8170a9-26a1-47e5-bfe0-291054d4036a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:10.917878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8170a9-26a1-47e5-bfe0-291054d4036a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:10.918155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8170a9-26a1-47e5-bfe0-291054d4036a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:10.918428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8170a9-26a1-47e5-bfe0-291054d4036a None admin] PATH_INFO: `/v3/domains/d9be7572b11441af8a9c6ad4ce88b31c` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:10.919567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8170a9-26a1-47e5-bfe0-291054d4036a None admin] RBAC: Authorizing `identity:get_domain(domain_id=d9be7572b11441af8a9c6ad4ce88b31c)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:10.920990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8170a9-26a1-47e5-bfe0-291054d4036a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:10.922554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 585/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Aug 31 09:37:10 2022] GET /identity/v3/domains/d9be7572b11441af8a9c6ad4ce88b31c => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:10.928949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ddaf52-1b85-4a2d-8d1c-824d7e80285e None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:11.033853 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ddaf52-1b85-4a2d-8d1c-824d7e80285e None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:11.071008 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbb762-ff3b-4f1e-b79c-7d5d42c2e406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:11.071676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbb762-ff3b-4f1e-b79c-7d5d42c2e406 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:11.071958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbb762-ff3b-4f1e-b79c-7d5d42c2e406 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:11.072232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbb762-ff3b-4f1e-b79c-7d5d42c2e406 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:11.072926 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbb762-ff3b-4f1e-b79c-7d5d42c2e406 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:11.074423 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbb762-ff3b-4f1e-b79c-7d5d42c2e406 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:11.083030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:11.097528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 592/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:37:10 2022] POST /identity/v3/users => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:37:13.078705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df546670-a141-4380-b39c-8aed660e0487 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:13.079203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df546670-a141-4380-b39c-8aed660e0487 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:13.079316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df546670-a141-4380-b39c-8aed660e0487 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:13.080617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 586/1178] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:13.087059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef78f7-ec5b-4c9c-999c-de4c7ac65326 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:13.087605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef78f7-ec5b-4c9c-999c-de4c7ac65326 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:13.087696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef78f7-ec5b-4c9c-999c-de4c7ac65326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:13.106704 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-a3ef78f7-ec5b-4c9c-999c-de4c7ac65326 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:13.107973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a3ef78f7-ec5b-4c9c-999c-de4c7ac65326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:13.181978 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 593/1179] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:13.455572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df546670-a141-4380-b39c-8aed660e0487 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:13.499009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:13.558236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df546670-a141-4380-b39c-8aed660e0487 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:13.591584 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c00b7d-7fe7-4a68-9f34-89728c442c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:13.592205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c00b7d-7fe7-4a68-9f34-89728c442c92 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:13.592483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c00b7d-7fe7-4a68-9f34-89728c442c92 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:13.592754 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c00b7d-7fe7-4a68-9f34-89728c442c92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:13.603067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c00b7d-7fe7-4a68-9f34-89728c442c92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-39c00b7d-7fe7-4a68-9f34-89728c442c92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:37:13.607820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:13.608979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:37:13.609893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 587/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:37:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:13.615151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef78f7-ec5b-4c9c-999c-de4c7ac65326 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:13.713530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef78f7-ec5b-4c9c-999c-de4c7ac65326 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:13.734017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:13.748599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9d9f6-234e-44b7-ac68-1388314be2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:13.749207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9d9f6-234e-44b7-ac68-1388314be2d5 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:13.749488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9d9f6-234e-44b7-ac68-1388314be2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:13.749758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9d9f6-234e-44b7-ac68-1388314be2d5 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:13.750423 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9d9f6-234e-44b7-ac68-1388314be2d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:13.750725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9d9f6-234e-44b7-ac68-1388314be2d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:13.752128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9d9f6-234e-44b7-ac68-1388314be2d5 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:13.760244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 594/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:37:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:13.766468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c00b7d-7fe7-4a68-9f34-89728c442c92 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:13.864136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:13.867305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c00b7d-7fe7-4a68-9f34-89728c442c92 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:13.902797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951d1df-8106-45cd-8de6-fbe146d28376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:13.903457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951d1df-8106-45cd-8de6-fbe146d28376 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:13.903736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951d1df-8106-45cd-8de6-fbe146d28376 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:13.904007 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951d1df-8106-45cd-8de6-fbe146d28376 None admin] PATH_INFO: `/v3/users/5828fb9cf3a346718c8d96638e7aff63` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:13.917441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951d1df-8106-45cd-8de6-fbe146d28376 None admin] RBAC: Authorizing `identity:get_user(user_id=5828fb9cf3a346718c8d96638e7aff63)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:13.918892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951d1df-8106-45cd-8de6-fbe146d28376 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:13.920565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 588/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:37:13 2022] GET /identity/v3/users/5828fb9cf3a346718c8d96638e7aff63 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:13.926473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9d9f6-234e-44b7-ac68-1388314be2d5 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:14.027592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9d9f6-234e-44b7-ac68-1388314be2d5 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:14.060024 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7166e7-989f-478b-9bb1-b7bcc771dab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:14.060692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7166e7-989f-478b-9bb1-b7bcc771dab3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:14.060911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7166e7-989f-478b-9bb1-b7bcc771dab3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:14.061184 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7166e7-989f-478b-9bb1-b7bcc771dab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:14.110268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:14.128682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7166e7-989f-478b-9bb1-b7bcc771dab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:14.130142 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7166e7-989f-478b-9bb1-b7bcc771dab3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:14.229313 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 595/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:37:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:37:14.234921 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951d1df-8106-45cd-8de6-fbe146d28376 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:14.337059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951d1df-8106-45cd-8de6-fbe146d28376 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:14.371816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9fe5d1-bd46-4d8a-9428-8c4d76d86dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:14.375486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fe5d1-bd46-4d8a-9428-8c4d76d86dbf None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:14.375791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fe5d1-bd46-4d8a-9428-8c4d76d86dbf None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:14.376073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fe5d1-bd46-4d8a-9428-8c4d76d86dbf None admin] PATH_INFO: `/v3/projects/a823e5db673a47bbb4e87bfcbda4ada2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:14.377240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9fe5d1-bd46-4d8a-9428-8c4d76d86dbf None admin] RBAC: Authorizing `identity:get_project(project_id=a823e5db673a47bbb4e87bfcbda4ada2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:14.378672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9fe5d1-bd46-4d8a-9428-8c4d76d86dbf None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:14.380335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 589/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:37:14 2022] GET /identity/v3/projects/a823e5db673a47bbb4e87bfcbda4ada2 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:14.387598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7166e7-989f-478b-9bb1-b7bcc771dab3 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:14.471439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:14.490067 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7166e7-989f-478b-9bb1-b7bcc771dab3 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:14.523201 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df7ed4-55f7-4124-b7f5-74f45ad5ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:14.523783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df7ed4-55f7-4124-b7f5-74f45ad5ad22 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:14.524059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df7ed4-55f7-4124-b7f5-74f45ad5ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:14.524327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df7ed4-55f7-4124-b7f5-74f45ad5ad22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:14.525038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df7ed4-55f7-4124-b7f5-74f45ad5ad22 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=a823e5db673a47bbb4e87bfcbda4ada2, user.id=5828fb9cf3a346718c8d96638e7aff63) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:14.525340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df7ed4-55f7-4124-b7f5-74f45ad5ad22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:14.526743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df7ed4-55f7-4124-b7f5-74f45ad5ad22 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:14.531997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 596/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:37:14 2022] GET /identity/v3/role_assignments?user.id=5828fb9cf3a346718c8d96638e7aff63&scope.project.id=a823e5db673a47bbb4e87bfcbda4ada2&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:16.426701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae369725-c665-42d2-8976-be3215099cb7 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:16.427282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae369725-c665-42d2-8976-be3215099cb7 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:16.427282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae369725-c665-42d2-8976-be3215099cb7 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:16.429282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 590/1186] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:37:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:16.434660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a529d-bdd1-4691-b80d-98072841934b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:16.435112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a529d-bdd1-4691-b80d-98072841934b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:16.435261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a529d-bdd1-4691-b80d-98072841934b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:16.458535 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e19a529d-bdd1-4691-b80d-98072841934b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:16.460403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e19a529d-bdd1-4691-b80d-98072841934b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:16.529838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 597/1187] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:16.806196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae369725-c665-42d2-8976-be3215099cb7 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:16.833659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:16.912625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae369725-c665-42d2-8976-be3215099cb7 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:16.948649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343e6e8-1bf7-459c-961a-a0f3a15c9050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:16.949266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343e6e8-1bf7-459c-961a-a0f3a15c9050 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:16.949535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343e6e8-1bf7-459c-961a-a0f3a15c9050 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:16.949804 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343e6e8-1bf7-459c-961a-a0f3a15c9050 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:16.960567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343e6e8-1bf7-459c-961a-a0f3a15c9050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-c343e6e8-1bf7-459c-961a-a0f3a15c9050 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:37:16.965206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:16.966392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:37:16.967540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 591/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:37:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:16.972551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a529d-bdd1-4691-b80d-98072841934b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:17.052272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:17.077710 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a529d-bdd1-4691-b80d-98072841934b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:17.111233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ffa74-598f-46ef-8842-ccf50e476fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:17.111826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ffa74-598f-46ef-8842-ccf50e476fe0 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:17.112101 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ffa74-598f-46ef-8842-ccf50e476fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:17.112365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ffa74-598f-46ef-8842-ccf50e476fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:17.113018 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ffa74-598f-46ef-8842-ccf50e476fe0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:17.113314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ffa74-598f-46ef-8842-ccf50e476fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:17.114688 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ffa74-598f-46ef-8842-ccf50e476fe0 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:17.123943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 598/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:37:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:17.129808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343e6e8-1bf7-459c-961a-a0f3a15c9050 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:17.217080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:17.237902 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343e6e8-1bf7-459c-961a-a0f3a15c9050 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:17.273467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294413ab-ce7b-4afa-bfdf-bf734b225c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:17.274089 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294413ab-ce7b-4afa-bfdf-bf734b225c25 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:17.274360 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294413ab-ce7b-4afa-bfdf-bf734b225c25 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:17.274625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294413ab-ce7b-4afa-bfdf-bf734b225c25 None admin] PATH_INFO: `/v3/users/5828fb9cf3a346718c8d96638e7aff63` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:17.275757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294413ab-ce7b-4afa-bfdf-bf734b225c25 None admin] RBAC: Authorizing `identity:get_user(user_id=5828fb9cf3a346718c8d96638e7aff63)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:17.277180 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294413ab-ce7b-4afa-bfdf-bf734b225c25 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:17.278738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 592/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:37:17 2022] GET /identity/v3/users/5828fb9cf3a346718c8d96638e7aff63 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:17.285244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ffa74-598f-46ef-8842-ccf50e476fe0 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:17.385242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ffa74-598f-46ef-8842-ccf50e476fe0 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:17.419260 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f588c-0b29-456f-8fe2-6dc7e09f5a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:17.419885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f588c-0b29-456f-8fe2-6dc7e09f5a3c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:17.420164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f588c-0b29-456f-8fe2-6dc7e09f5a3c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:17.420438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f588c-0b29-456f-8fe2-6dc7e09f5a3c None admin] PATH_INFO: `/v3/projects/a823e5db673a47bbb4e87bfcbda4ada2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:17.421570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f588c-0b29-456f-8fe2-6dc7e09f5a3c None admin] RBAC: Authorizing `identity:get_project(project_id=a823e5db673a47bbb4e87bfcbda4ada2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:17.422996 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f588c-0b29-456f-8fe2-6dc7e09f5a3c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:17.424675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 599/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:37:17 2022] GET /identity/v3/projects/a823e5db673a47bbb4e87bfcbda4ada2 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:17.431588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294413ab-ce7b-4afa-bfdf-bf734b225c25 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:17.537166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294413ab-ce7b-4afa-bfdf-bf734b225c25 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:17.572485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996a397-084e-4a72-a208-b8108c8531d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:17.573134 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996a397-084e-4a72-a208-b8108c8531d9 None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:17.573405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996a397-084e-4a72-a208-b8108c8531d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:17.573669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996a397-084e-4a72-a208-b8108c8531d9 None admin] PATH_INFO: `/v3/projects/a823e5db673a47bbb4e87bfcbda4ada2/users/5828fb9cf3a346718c8d96638e7aff63/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:17.575592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996a397-084e-4a72-a208-b8108c8531d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a823e5db673a47bbb4e87bfcbda4ada2, user_id=5828fb9cf3a346718c8d96638e7aff63, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:17.577012 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996a397-084e-4a72-a208-b8108c8531d9 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:17.591691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 593/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:37:17 2022] PUT /identity/v3/projects/a823e5db673a47bbb4e87bfcbda4ada2/users/5828fb9cf3a346718c8d96638e7aff63/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:37:19.517585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87609d-d93a-4590-8ac0-9f0f089d4c05 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:19.518119 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87609d-d93a-4590-8ac0-9f0f089d4c05 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:19.518119 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87609d-d93a-4590-8ac0-9f0f089d4c05 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:19.519520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 600/1193] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:19.525327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6f771-5b95-42ae-8e4a-6d9786896219 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:19.525758 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6f771-5b95-42ae-8e4a-6d9786896219 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:19.525872 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6f771-5b95-42ae-8e4a-6d9786896219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:19.544650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-77d6f771-5b95-42ae-8e4a-6d9786896219 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:19.545849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-77d6f771-5b95-42ae-8e4a-6d9786896219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:19.555315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:19.619025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 594/1194] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:19.889702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87609d-d93a-4590-8ac0-9f0f089d4c05 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:19.899651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:19.991319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87609d-d93a-4590-8ac0-9f0f089d4c05 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:20.024262 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d037d36-b7a8-42e0-bb6e-7e84f7e13d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:20.024892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d037d36-b7a8-42e0-bb6e-7e84f7e13d4a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:20.025169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d037d36-b7a8-42e0-bb6e-7e84f7e13d4a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:20.025439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d037d36-b7a8-42e0-bb6e-7e84f7e13d4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:20.035864 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d037d36-b7a8-42e0-bb6e-7e84f7e13d4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-3d037d36-b7a8-42e0-bb6e-7e84f7e13d4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:37:20.041128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:20.042259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:37:20.043556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 601/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:37:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:20.048257 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6f771-5b95-42ae-8e4a-6d9786896219 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:20.144043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6f771-5b95-42ae-8e4a-6d9786896219 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:20.176954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9caf1-1c57-45eb-b64b-0da3d7d18456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:20.177549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9caf1-1c57-45eb-b64b-0da3d7d18456 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:20.177823 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9caf1-1c57-45eb-b64b-0da3d7d18456 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:20.178807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9caf1-1c57-45eb-b64b-0da3d7d18456 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:20.179481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9caf1-1c57-45eb-b64b-0da3d7d18456 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:20.179779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9caf1-1c57-45eb-b64b-0da3d7d18456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:20.181143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9caf1-1c57-45eb-b64b-0da3d7d18456 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:20.189109 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 595/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:37:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:20.195517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d037d36-b7a8-42e0-bb6e-7e84f7e13d4a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:20.262010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:20.300341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d037d36-b7a8-42e0-bb6e-7e84f7e13d4a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:20.332729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bff430-047d-411d-8ee1-5917fe2a0d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:20.333344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bff430-047d-411d-8ee1-5917fe2a0d7d None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:20.333621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bff430-047d-411d-8ee1-5917fe2a0d7d None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:20.333890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bff430-047d-411d-8ee1-5917fe2a0d7d None admin] PATH_INFO: `/v3/users/5828fb9cf3a346718c8d96638e7aff63` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:20.335638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bff430-047d-411d-8ee1-5917fe2a0d7d None admin] RBAC: Authorizing `identity:get_user(user_id=5828fb9cf3a346718c8d96638e7aff63)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:20.337053 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bff430-047d-411d-8ee1-5917fe2a0d7d None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:20.338640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 602/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 09:37:20 2022] GET /identity/v3/users/5828fb9cf3a346718c8d96638e7aff63 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:20.344601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9caf1-1c57-45eb-b64b-0da3d7d18456 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:20.355165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:20.454941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9caf1-1c57-45eb-b64b-0da3d7d18456 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:20.492407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97d75a-25a7-40b6-8e02-f40f591d1144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:20.493006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97d75a-25a7-40b6-8e02-f40f591d1144 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:20.493277 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97d75a-25a7-40b6-8e02-f40f591d1144 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:20.493538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97d75a-25a7-40b6-8e02-f40f591d1144 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:20.567405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97d75a-25a7-40b6-8e02-f40f591d1144 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:20.569601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97d75a-25a7-40b6-8e02-f40f591d1144 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:20.578041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:20.672761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 596/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:37:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:37:20.679458 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bff430-047d-411d-8ee1-5917fe2a0d7d None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:20.788397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bff430-047d-411d-8ee1-5917fe2a0d7d None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:20.823689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f807a60-a80f-40ee-844a-e2281645d59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:20.824337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f807a60-a80f-40ee-844a-e2281645d59c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:20.824617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f807a60-a80f-40ee-844a-e2281645d59c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:20.824888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f807a60-a80f-40ee-844a-e2281645d59c None admin] PATH_INFO: `/v3/projects/a823e5db673a47bbb4e87bfcbda4ada2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:20.826104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f807a60-a80f-40ee-844a-e2281645d59c None admin] RBAC: Authorizing `identity:get_project(project_id=a823e5db673a47bbb4e87bfcbda4ada2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:20.827583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f807a60-a80f-40ee-844a-e2281645d59c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:20.829248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 603/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 09:37:20 2022] GET /identity/v3/projects/a823e5db673a47bbb4e87bfcbda4ada2 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:20.836076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97d75a-25a7-40b6-8e02-f40f591d1144 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:20.942826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97d75a-25a7-40b6-8e02-f40f591d1144 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:20.950954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:20.982288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933e972-929d-44d6-8853-d07a9895d2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:20.982889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933e972-929d-44d6-8853-d07a9895d2a6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:20.983191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933e972-929d-44d6-8853-d07a9895d2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:20.983459 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933e972-929d-44d6-8853-d07a9895d2a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:20.984176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933e972-929d-44d6-8853-d07a9895d2a6 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=a823e5db673a47bbb4e87bfcbda4ada2, user.id=5828fb9cf3a346718c8d96638e7aff63) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:20.984483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933e972-929d-44d6-8853-d07a9895d2a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:20.985914 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933e972-929d-44d6-8853-d07a9895d2a6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:20.992337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 597/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:37:20 2022] GET /identity/v3/role_assignments?user.id=5828fb9cf3a346718c8d96638e7aff63&scope.project.id=a823e5db673a47bbb4e87bfcbda4ada2&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:30.818303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:30.989122 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=15 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:19.292517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026311fd-92bf-4d48-a5c7-7900d1bebce6 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:19.295162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026311fd-92bf-4d48-a5c7-7900d1bebce6 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:19.295162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026311fd-92bf-4d48-a5c7-7900d1bebce6 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:19.298507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 604/1201] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:38:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:38:19.304920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c1f0e-756d-4776-94dc-c60b59c037c3 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:19.305514 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c1f0e-756d-4776-94dc-c60b59c037c3 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:19.305737 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c1f0e-756d-4776-94dc-c60b59c037c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:19.338534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-1d3c1f0e-756d-4776-94dc-c60b59c037c3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:19.339898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1d3c1f0e-756d-4776-94dc-c60b59c037c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:19.452309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 598/1202] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:38:19.735545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026311fd-92bf-4d48-a5c7-7900d1bebce6 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:19.916861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026311fd-92bf-4d48-a5c7-7900d1bebce6 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:19.971425 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6efe93-0a24-43dc-98be-a79a8b42c6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:19.972206 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6efe93-0a24-43dc-98be-a79a8b42c6cc None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:19.972485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6efe93-0a24-43dc-98be-a79a8b42c6cc None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:19.973824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6efe93-0a24-43dc-98be-a79a8b42c6cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:19.988895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6efe93-0a24-43dc-98be-a79a8b42c6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:19.990455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6efe93-0a24-43dc-98be-a79a8b42c6cc None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-eb6efe93-0a24-43dc-98be-a79a8b42c6cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:20.009568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:20.016251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:20.018235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:20.018235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:20.018235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:20.018235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 605/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:38:20.018723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c1f0e-756d-4776-94dc-c60b59c037c3 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:20.194277 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c1f0e-756d-4776-94dc-c60b59c037c3 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:20.241163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:20.248619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038b3b9-b724-4459-a77b-cdb06acb3807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:20.249848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038b3b9-b724-4459-a77b-cdb06acb3807 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:20.250199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038b3b9-b724-4459-a77b-cdb06acb3807 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:20.250446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038b3b9-b724-4459-a77b-cdb06acb3807 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:20.251292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038b3b9-b724-4459-a77b-cdb06acb3807 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:20.251677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038b3b9-b724-4459-a77b-cdb06acb3807 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:20.255193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038b3b9-b724-4459-a77b-cdb06acb3807 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:20.277278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 599/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:20.287014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6efe93-0a24-43dc-98be-a79a8b42c6cc None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:20.369063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:20.447222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6efe93-0a24-43dc-98be-a79a8b42c6cc None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:20.507355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa7de7-6749-4a9d-b710-183a4a9b6acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:20.514789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa7de7-6749-4a9d-b710-183a4a9b6acf None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:20.514789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa7de7-6749-4a9d-b710-183a4a9b6acf None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:20.514789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa7de7-6749-4a9d-b710-183a4a9b6acf None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:20.514789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa7de7-6749-4a9d-b710-183a4a9b6acf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:20.514789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa7de7-6749-4a9d-b710-183a4a9b6acf None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:20.545957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 606/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 09:38:20 2022] POST /identity/v3/users => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:38:23.087324 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446a5c9-fbfa-4f8d-822d-57c6d3676e23 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:23.087885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446a5c9-fbfa-4f8d-822d-57c6d3676e23 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:23.088275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446a5c9-fbfa-4f8d-822d-57c6d3676e23 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:23.090815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 600/1206] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:38:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:23.101009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419162e-72ce-4a7f-963c-74ad2789639b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:23.101573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419162e-72ce-4a7f-963c-74ad2789639b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:23.101976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419162e-72ce-4a7f-963c-74ad2789639b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:23.134138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-c419162e-72ce-4a7f-963c-74ad2789639b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:23.135985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c419162e-72ce-4a7f-963c-74ad2789639b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:23.268090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 607/1207] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:23.554659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446a5c9-fbfa-4f8d-822d-57c6d3676e23 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:23.703498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446a5c9-fbfa-4f8d-822d-57c6d3676e23 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:23.778689 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54019366-c7e7-4e5b-8aef-a4391a345681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:23.779949 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54019366-c7e7-4e5b-8aef-a4391a345681 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:23.780326 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54019366-c7e7-4e5b-8aef-a4391a345681 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:23.780730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54019366-c7e7-4e5b-8aef-a4391a345681 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:23.798256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54019366-c7e7-4e5b-8aef-a4391a345681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-54019366-c7e7-4e5b-8aef-a4391a345681 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:38:23.808246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:23.810177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:23.811032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 601/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:38:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:23.817363 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419162e-72ce-4a7f-963c-74ad2789639b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:23.879994 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:23.982743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419162e-72ce-4a7f-963c-74ad2789639b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:24.033313 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a21d24-efdf-45e2-9f97-7e16afd0d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:24.033978 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a21d24-efdf-45e2-9f97-7e16afd0d5fb None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:24.034441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a21d24-efdf-45e2-9f97-7e16afd0d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:24.034597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a21d24-efdf-45e2-9f97-7e16afd0d5fb None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:24.036362 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a21d24-efdf-45e2-9f97-7e16afd0d5fb None admin] RBAC: Adding query filter params (name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:24.036744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a21d24-efdf-45e2-9f97-7e16afd0d5fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:24.038488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a21d24-efdf-45e2-9f97-7e16afd0d5fb None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:24.048975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 608/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:38:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:24.056253 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54019366-c7e7-4e5b-8aef-a4391a345681 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:24.132841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:24.212461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54019366-c7e7-4e5b-8aef-a4391a345681 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:24.275536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb98f82-8930-479e-9287-a8bd68065d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:24.276479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98f82-8930-479e-9287-a8bd68065d50 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:24.280298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98f82-8930-479e-9287-a8bd68065d50 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:24.280298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98f82-8930-479e-9287-a8bd68065d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:24.297230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb98f82-8930-479e-9287-a8bd68065d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:24.299963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb98f82-8930-479e-9287-a8bd68065d50 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-7bb98f82-8930-479e-9287-a8bd68065d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:24.324672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:24.326163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:24.327855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:24.327855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:24.327855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:24.327855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 602/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:24.333664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a21d24-efdf-45e2-9f97-7e16afd0d5fb None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:24.467197 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:24.471196 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a21d24-efdf-45e2-9f97-7e16afd0d5fb None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:24.517449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6e73c-1ccf-4312-a0a5-c6f660f9f378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:24.518219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6e73c-1ccf-4312-a0a5-c6f660f9f378 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:24.518682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6e73c-1ccf-4312-a0a5-c6f660f9f378 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:24.519086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6e73c-1ccf-4312-a0a5-c6f660f9f378 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:24.520045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6e73c-1ccf-4312-a0a5-c6f660f9f378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:24.520440 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6e73c-1ccf-4312-a0a5-c6f660f9f378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:24.522091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6e73c-1ccf-4312-a0a5-c6f660f9f378 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:24.543645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 609/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:24.544879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb98f82-8930-479e-9287-a8bd68065d50 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:24.698580 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb98f82-8930-479e-9287-a8bd68065d50 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:24.701027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:24.747770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf194c-0336-4522-ad52-1af40dee67d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:24.748438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf194c-0336-4522-ad52-1af40dee67d2 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:24.748754 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf194c-0336-4522-ad52-1af40dee67d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:24.749028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf194c-0336-4522-ad52-1af40dee67d2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:24.758364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf194c-0336-4522-ad52-1af40dee67d2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:24.759938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf194c-0336-4522-ad52-1af40dee67d2 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-5acf194c-0336-4522-ad52-1af40dee67d2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:38:24.773848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:24.776210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:24.778156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:24.778156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:24.778156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:38:24.778156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:38:24.778156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:38:24.778156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:38:24.778156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:38:24.778156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:38:24.778156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:24.778156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:24.778156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 603/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:24.787108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6e73c-1ccf-4312-a0a5-c6f660f9f378 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:24.987691 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6e73c-1ccf-4312-a0a5-c6f660f9f378 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:25.040054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f36067-5ed5-47a5-be66-769305463650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:25.040908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f36067-5ed5-47a5-be66-769305463650 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:25.041332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f36067-5ed5-47a5-be66-769305463650 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:25.041653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f36067-5ed5-47a5-be66-769305463650 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:25.042716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f36067-5ed5-47a5-be66-769305463650 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:25.042938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f36067-5ed5-47a5-be66-769305463650 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:25.044819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f36067-5ed5-47a5-be66-769305463650 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:25.069262 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 610/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Aug 31 09:38:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:25.075930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf194c-0336-4522-ad52-1af40dee67d2 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:25.248343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf194c-0336-4522-ad52-1af40dee67d2 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:25.304584 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242dad2-191a-4f69-a5cc-a391c7e2a0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:25.305577 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242dad2-191a-4f69-a5cc-a391c7e2a0a8 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:25.305979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242dad2-191a-4f69-a5cc-a391c7e2a0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:25.306464 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242dad2-191a-4f69-a5cc-a391c7e2a0a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:25.360728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:25.420923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242dad2-191a-4f69-a5cc-a391c7e2a0a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:25.424205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242dad2-191a-4f69-a5cc-a391c7e2a0a8 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:25.584725 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 604/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:38:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:38:25.586437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f36067-5ed5-47a5-be66-769305463650 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:25.617600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:25.739392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f36067-5ed5-47a5-be66-769305463650 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:25.784951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9912c-ffbb-4a0f-8a9f-0303411a26c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:25.786387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9912c-ffbb-4a0f-8a9f-0303411a26c3 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:25.786841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9912c-ffbb-4a0f-8a9f-0303411a26c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:25.787382 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9912c-ffbb-4a0f-8a9f-0303411a26c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:25.802713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9912c-ffbb-4a0f-8a9f-0303411a26c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:25.804566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9912c-ffbb-4a0f-8a9f-0303411a26c3 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-eed9912c-ffbb-4a0f-8a9f-0303411a26c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:25.824084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:25.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:25.834610 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:25.834610 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:25.834610 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:25.834610 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 611/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:25.835076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242dad2-191a-4f69-a5cc-a391c7e2a0a8 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:25.943526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:26.008542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242dad2-191a-4f69-a5cc-a391c7e2a0a8 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:26.057495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464b4e5-e334-4b56-b512-5d69a31589c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:26.057495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464b4e5-e334-4b56-b512-5d69a31589c6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:26.057495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464b4e5-e334-4b56-b512-5d69a31589c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:26.057495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464b4e5-e334-4b56-b512-5d69a31589c6 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:26.058344 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464b4e5-e334-4b56-b512-5d69a31589c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:26.058873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464b4e5-e334-4b56-b512-5d69a31589c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:26.061498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464b4e5-e334-4b56-b512-5d69a31589c6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:26.091314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 605/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:26.096850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9912c-ffbb-4a0f-8a9f-0303411a26c3 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:26.201723 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:26.241202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9912c-ffbb-4a0f-8a9f-0303411a26c3 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:26.289373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b8250-cee2-4f5e-adf4-b789dc9a5a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:26.290169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b8250-cee2-4f5e-adf4-b789dc9a5a69 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:26.290521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b8250-cee2-4f5e-adf4-b789dc9a5a69 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:26.290845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b8250-cee2-4f5e-adf4-b789dc9a5a69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:26.304327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b8250-cee2-4f5e-adf4-b789dc9a5a69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:26.306296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b8250-cee2-4f5e-adf4-b789dc9a5a69 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-885b8250-cee2-4f5e-adf4-b789dc9a5a69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:38:26.330472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:26.331528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:26.333794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:38:26.333794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:38:26.333794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:38:26.333794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:38:26.333794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:38:26.333794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:38:26.333794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:26.333794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:26.333794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 612/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:38:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:26.340642 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464b4e5-e334-4b56-b512-5d69a31589c6 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:26.471644 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464b4e5-e334-4b56-b512-5d69a31589c6 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:26.498782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:26.515163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779433ce-2533-404e-baeb-31b218a5dc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:26.516078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779433ce-2533-404e-baeb-31b218a5dc9d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:26.516540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779433ce-2533-404e-baeb-31b218a5dc9d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:26.517021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779433ce-2533-404e-baeb-31b218a5dc9d None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:26.518282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779433ce-2533-404e-baeb-31b218a5dc9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:26.518822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779433ce-2533-404e-baeb-31b218a5dc9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:26.522506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779433ce-2533-404e-baeb-31b218a5dc9d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:26.544619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 606/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:26.552439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b8250-cee2-4f5e-adf4-b789dc9a5a69 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:26.692349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b8250-cee2-4f5e-adf4-b789dc9a5a69 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:26.733899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:26.747620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bccd2-cbdb-42af-a037-82d45dad8400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:26.748621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bccd2-cbdb-42af-a037-82d45dad8400 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:26.749204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bccd2-cbdb-42af-a037-82d45dad8400 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:26.749689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bccd2-cbdb-42af-a037-82d45dad8400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:26.750626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bccd2-cbdb-42af-a037-82d45dad8400 None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=560df3eb05f14b6489158ab053f282ca) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:26.751199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bccd2-cbdb-42af-a037-82d45dad8400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:26.753076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bccd2-cbdb-42af-a037-82d45dad8400 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:26.761041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 613/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:38:26 2022] GET /identity/v3/role_assignments?user.id=560df3eb05f14b6489158ab053f282ca&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:28.977104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff2570-aaf7-4896-b006-bee9b3eb49cf None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:28.977840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff2570-aaf7-4896-b006-bee9b3eb49cf None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:28.977840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff2570-aaf7-4896-b006-bee9b3eb49cf None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:28.979102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 607/1220] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:28.985014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9797a7-1d2c-412d-a829-3c438f0709fc None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:28.985624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9797a7-1d2c-412d-a829-3c438f0709fc None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:28.985624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9797a7-1d2c-412d-a829-3c438f0709fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:29.009722 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-bf9797a7-1d2c-412d-a829-3c438f0709fc None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:29.011397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-bf9797a7-1d2c-412d-a829-3c438f0709fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:29.119715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 614/1221] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:29.418736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff2570-aaf7-4896-b006-bee9b3eb49cf None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:29.565256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff2570-aaf7-4896-b006-bee9b3eb49cf None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:29.622420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dae4d-9a12-44a2-982e-22409d7d5903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:29.623271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dae4d-9a12-44a2-982e-22409d7d5903 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:29.623561 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dae4d-9a12-44a2-982e-22409d7d5903 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:29.623825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dae4d-9a12-44a2-982e-22409d7d5903 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:29.638771 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4dae4d-9a12-44a2-982e-22409d7d5903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-1f4dae4d-9a12-44a2-982e-22409d7d5903 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:38:29.646961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:29.648670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:29.650743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 608/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:38:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:29.655373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9797a7-1d2c-412d-a829-3c438f0709fc None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:29.848114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9797a7-1d2c-412d-a829-3c438f0709fc None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:29.882877 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:29.897760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380eefe-4a64-4629-b0f1-df1d0c01ec4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:29.898720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380eefe-4a64-4629-b0f1-df1d0c01ec4a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:29.899097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380eefe-4a64-4629-b0f1-df1d0c01ec4a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:29.899657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380eefe-4a64-4629-b0f1-df1d0c01ec4a None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:29.900371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380eefe-4a64-4629-b0f1-df1d0c01ec4a None admin] RBAC: Adding query filter params (name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:29.900717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380eefe-4a64-4629-b0f1-df1d0c01ec4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:29.902229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380eefe-4a64-4629-b0f1-df1d0c01ec4a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:29.912166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 615/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:38:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:29.918713 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dae4d-9a12-44a2-982e-22409d7d5903 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:29.990597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:30.070957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dae4d-9a12-44a2-982e-22409d7d5903 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:30.111986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff3303-3057-4d5f-9222-1d6e44cbe860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:30.112612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff3303-3057-4d5f-9222-1d6e44cbe860 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:30.113870 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff3303-3057-4d5f-9222-1d6e44cbe860 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:30.114026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff3303-3057-4d5f-9222-1d6e44cbe860 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:30.131158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff3303-3057-4d5f-9222-1d6e44cbe860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:30.133105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff3303-3057-4d5f-9222-1d6e44cbe860 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-aeff3303-3057-4d5f-9222-1d6e44cbe860 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:30.150570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:30.151694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:30.152592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:30.152592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:30.152592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:30.152592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 609/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:30.158421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380eefe-4a64-4629-b0f1-df1d0c01ec4a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:30.341634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380eefe-4a64-4629-b0f1-df1d0c01ec4a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:30.391298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2e4fe-dba5-473e-a89f-714cd39da875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:30.391828 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2e4fe-dba5-473e-a89f-714cd39da875 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:30.392108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2e4fe-dba5-473e-a89f-714cd39da875 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:30.392372 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2e4fe-dba5-473e-a89f-714cd39da875 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:30.393020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2e4fe-dba5-473e-a89f-714cd39da875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:30.393319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2e4fe-dba5-473e-a89f-714cd39da875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:30.395328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2e4fe-dba5-473e-a89f-714cd39da875 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:30.409742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 616/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:30.416272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff3303-3057-4d5f-9222-1d6e44cbe860 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:30.586151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:30.600045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff3303-3057-4d5f-9222-1d6e44cbe860 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:30.648111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a4ad2-789b-40f8-b0e4-fc22b162b16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:30.649038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a4ad2-789b-40f8-b0e4-fc22b162b16b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:30.649500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a4ad2-789b-40f8-b0e4-fc22b162b16b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:30.649954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a4ad2-789b-40f8-b0e4-fc22b162b16b None admin] PATH_INFO: `/v3/users/placement` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:30.659833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a4ad2-789b-40f8-b0e4-fc22b162b16b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:30.661617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a4ad2-789b-40f8-b0e4-fc22b162b16b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-088a4ad2-789b-40f8-b0e4-fc22b162b16b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:38:30.674852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:30.676653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:30.677593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:30.677593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:30.677593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:38:30.677593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:38:30.677593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:38:30.677593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:38:30.677593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:38:30.677593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:38:30.677593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:30.677593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:30.677593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 610/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:30.683325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2e4fe-dba5-473e-a89f-714cd39da875 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:30.761939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:30.844137 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2e4fe-dba5-473e-a89f-714cd39da875 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:30.894830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f32bdd-7a37-4da0-9a94-bd82dffdef40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:30.896288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f32bdd-7a37-4da0-9a94-bd82dffdef40 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:30.896804 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f32bdd-7a37-4da0-9a94-bd82dffdef40 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:30.897308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f32bdd-7a37-4da0-9a94-bd82dffdef40 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:30.899071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f32bdd-7a37-4da0-9a94-bd82dffdef40 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:30.899613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f32bdd-7a37-4da0-9a94-bd82dffdef40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:30.902883 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f32bdd-7a37-4da0-9a94-bd82dffdef40 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:30.935193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 617/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Aug 31 09:38:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:30.943389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a4ad2-789b-40f8-b0e4-fc22b162b16b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:31.099851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a4ad2-789b-40f8-b0e4-fc22b162b16b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:31.150850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140b541-e305-45a6-8fd3-a791c9d0034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:31.151751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140b541-e305-45a6-8fd3-a791c9d0034a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:31.152108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140b541-e305-45a6-8fd3-a791c9d0034a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:31.152525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140b541-e305-45a6-8fd3-a791c9d0034a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:31.171386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140b541-e305-45a6-8fd3-a791c9d0034a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:31.172986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140b541-e305-45a6-8fd3-a791c9d0034a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-6140b541-e305-45a6-8fd3-a791c9d0034a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:31.195286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:31.196466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:31.197466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:31.197466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:31.197466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:31.197466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 611/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:31.204231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f32bdd-7a37-4da0-9a94-bd82dffdef40 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:31.354510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f32bdd-7a37-4da0-9a94-bd82dffdef40 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:31.357476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:31.404144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f63d00-d302-4862-87bb-719dc1c4b448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:31.405020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f63d00-d302-4862-87bb-719dc1c4b448 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:31.405565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f63d00-d302-4862-87bb-719dc1c4b448 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:31.406116 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f63d00-d302-4862-87bb-719dc1c4b448 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:31.406948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f63d00-d302-4862-87bb-719dc1c4b448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:31.407327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f63d00-d302-4862-87bb-719dc1c4b448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:31.409604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f63d00-d302-4862-87bb-719dc1c4b448 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:31.428436 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 618/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:31.435596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140b541-e305-45a6-8fd3-a791c9d0034a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:31.450092 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:31.591339 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140b541-e305-45a6-8fd3-a791c9d0034a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:31.642965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddfb2b-1b88-479e-b572-bd8f1c28a1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:31.643576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddfb2b-1b88-479e-b572-bd8f1c28a1cc None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:31.643792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddfb2b-1b88-479e-b572-bd8f1c28a1cc None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:31.644723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddfb2b-1b88-479e-b572-bd8f1c28a1cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:31.652976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddfb2b-1b88-479e-b572-bd8f1c28a1cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:31.654718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddfb2b-1b88-479e-b572-bd8f1c28a1cc None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-7fddfb2b-1b88-479e-b572-bd8f1c28a1cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:38:31.668845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:31.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:31.677200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:38:31.677200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:38:31.677200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:38:31.677200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:38:31.677200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:38:31.677200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:38:31.677200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:31.677200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:31.677200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 612/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:38:31.681462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f63d00-d302-4862-87bb-719dc1c4b448 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:31.821541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f63d00-d302-4862-87bb-719dc1c4b448 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:31.865756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0120793-cd21-401c-ab0a-a06e7deecc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:31.866751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0120793-cd21-401c-ab0a-a06e7deecc70 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:31.867163 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0120793-cd21-401c-ab0a-a06e7deecc70 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:31.867540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0120793-cd21-401c-ab0a-a06e7deecc70 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:31.868441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0120793-cd21-401c-ab0a-a06e7deecc70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:31.868933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0120793-cd21-401c-ab0a-a06e7deecc70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:31.871562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0120793-cd21-401c-ab0a-a06e7deecc70 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:31.897267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 619/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:31.906867 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddfb2b-1b88-479e-b572-bd8f1c28a1cc None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:32.015963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:32.071717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddfb2b-1b88-479e-b572-bd8f1c28a1cc None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:32.124742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e51d75-c1d5-478e-ba19-afa24a81413a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:32.125470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e51d75-c1d5-478e-ba19-afa24a81413a None admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:32.125686 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e51d75-c1d5-478e-ba19-afa24a81413a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:32.125939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e51d75-c1d5-478e-ba19-afa24a81413a None admin] PATH_INFO: `/v3/projects/8ef8b586382941408cd88ee654c42d32/users/560df3eb05f14b6489158ab053f282ca/roles/dcba9d07b31a4387b3014365e6e2ea10` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:32.145903 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e51d75-c1d5-478e-ba19-afa24a81413a None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef8b586382941408cd88ee654c42d32, user_id=560df3eb05f14b6489158ab053f282ca, role_id=dcba9d07b31a4387b3014365e6e2ea10)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:32.147513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e51d75-c1d5-478e-ba19-afa24a81413a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:32.162786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 613/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:38:31 2022] PUT /identity/v3/projects/8ef8b586382941408cd88ee654c42d32/users/560df3eb05f14b6489158ab053f282ca/roles/dcba9d07b31a4387b3014365e6e2ea10 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:38:34.259203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dacd8-308f-40bc-895e-e8e2aa964429 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:34.259203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dacd8-308f-40bc-895e-e8e2aa964429 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:34.259203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dacd8-308f-40bc-895e-e8e2aa964429 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:34.260707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 620/1233] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:34.266782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10fc86-3c17-493d-bb11-216b6e1ac059 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:34.267649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10fc86-3c17-493d-bb11-216b6e1ac059 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:34.268112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10fc86-3c17-493d-bb11-216b6e1ac059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:34.289698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4b10fc86-3c17-493d-bb11-216b6e1ac059 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:34.291312 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4b10fc86-3c17-493d-bb11-216b6e1ac059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:34.366795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 614/1234] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:34.634923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dacd8-308f-40bc-895e-e8e2aa964429 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:34.665191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:34.749456 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dacd8-308f-40bc-895e-e8e2aa964429 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:34.787441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06755-024c-4e57-990d-e881cad27577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:34.788081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06755-024c-4e57-990d-e881cad27577 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:34.788361 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06755-024c-4e57-990d-e881cad27577 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:34.788635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06755-024c-4e57-990d-e881cad27577 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:34.798910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b06755-024c-4e57-990d-e881cad27577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-d2b06755-024c-4e57-990d-e881cad27577 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:38:34.803993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:34.804962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:34.805815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 621/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 09:38:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:34.811298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10fc86-3c17-493d-bb11-216b6e1ac059 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:34.863540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:34.932631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10fc86-3c17-493d-bb11-216b6e1ac059 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:34.974889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5ca10-caac-4852-9285-80fb485cbd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:34.975533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5ca10-caac-4852-9285-80fb485cbd9e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:34.975813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5ca10-caac-4852-9285-80fb485cbd9e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:34.976084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5ca10-caac-4852-9285-80fb485cbd9e None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:34.976752 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5ca10-caac-4852-9285-80fb485cbd9e None admin] RBAC: Adding query filter params (name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:34.977126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5ca10-caac-4852-9285-80fb485cbd9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:34.978590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5ca10-caac-4852-9285-80fb485cbd9e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:34.990616 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 615/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 09:38:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:34.998213 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06755-024c-4e57-990d-e881cad27577 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:35.103502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:35.122099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06755-024c-4e57-990d-e881cad27577 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:35.157719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf332279-1426-481d-ad0c-39376ebde360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:35.158435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf332279-1426-481d-ad0c-39376ebde360 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:35.158715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf332279-1426-481d-ad0c-39376ebde360 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:35.158988 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf332279-1426-481d-ad0c-39376ebde360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:35.169394 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf332279-1426-481d-ad0c-39376ebde360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:35.170840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf332279-1426-481d-ad0c-39376ebde360 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-bf332279-1426-481d-ad0c-39376ebde360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:35.186026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:35.187650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:35.188844 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:35.188844 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:35.188844 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:35.188844 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 622/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:35.194269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5ca10-caac-4852-9285-80fb485cbd9e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:35.312013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5ca10-caac-4852-9285-80fb485cbd9e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:35.322048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:35.347693 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf11fb-2b39-49b0-8c3a-2a1a3890f81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:35.348288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf11fb-2b39-49b0-8c3a-2a1a3890f81b None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:35.348570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf11fb-2b39-49b0-8c3a-2a1a3890f81b None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:35.348821 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf11fb-2b39-49b0-8c3a-2a1a3890f81b None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:35.349452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf11fb-2b39-49b0-8c3a-2a1a3890f81b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:35.349746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf11fb-2b39-49b0-8c3a-2a1a3890f81b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:35.351110 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf11fb-2b39-49b0-8c3a-2a1a3890f81b None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:35.363795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 616/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:35.369957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf332279-1426-481d-ad0c-39376ebde360 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:35.483863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf332279-1426-481d-ad0c-39376ebde360 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:35.522297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66be41f-df5e-4255-b46d-7e14b233be81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:35.523221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66be41f-df5e-4255-b46d-7e14b233be81 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:35.523501 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66be41f-df5e-4255-b46d-7e14b233be81 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:35.523842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66be41f-df5e-4255-b46d-7e14b233be81 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:35.527341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:35.533017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66be41f-df5e-4255-b46d-7e14b233be81 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:35.535477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66be41f-df5e-4255-b46d-7e14b233be81 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-b66be41f-df5e-4255-b46d-7e14b233be81 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:38:35.547395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:35.548478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:35.549591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:35.549591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:35.549591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:38:35.549591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:38:35.549591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:38:35.549591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:38:35.549591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:38:35.549591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:38:35.549591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:35.549591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:35.549591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 623/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:35.554601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf11fb-2b39-49b0-8c3a-2a1a3890f81b None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:35.666014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf11fb-2b39-49b0-8c3a-2a1a3890f81b None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:35.703631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdb3db-caec-4caa-adad-8cf125a10245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:35.704341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdb3db-caec-4caa-adad-8cf125a10245 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:35.704693 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdb3db-caec-4caa-adad-8cf125a10245 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:35.705033 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdb3db-caec-4caa-adad-8cf125a10245 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:35.705835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdb3db-caec-4caa-adad-8cf125a10245 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:35.706136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdb3db-caec-4caa-adad-8cf125a10245 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:35.707904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdb3db-caec-4caa-adad-8cf125a10245 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:35.726791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 617/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Aug 31 09:38:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:35.732936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66be41f-df5e-4255-b46d-7e14b233be81 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:35.840477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66be41f-df5e-4255-b46d-7e14b233be81 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:35.876770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecd6e8-7ffe-4745-a981-c0d87962a094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:35.877382 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd6e8-7ffe-4745-a981-c0d87962a094 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:35.877665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd6e8-7ffe-4745-a981-c0d87962a094 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:35.878030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd6e8-7ffe-4745-a981-c0d87962a094 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:35.966635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:35.978060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecd6e8-7ffe-4745-a981-c0d87962a094 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:35.980357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecd6e8-7ffe-4745-a981-c0d87962a094 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:36.117613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 624/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:38:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:38:36.123419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdb3db-caec-4caa-adad-8cf125a10245 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:36.153224 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:36.236295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdb3db-caec-4caa-adad-8cf125a10245 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:36.278083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9924e73-7b8d-408b-b356-5a6fcecb1981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:36.278735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9924e73-7b8d-408b-b356-5a6fcecb1981 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:36.279056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9924e73-7b8d-408b-b356-5a6fcecb1981 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:36.279311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9924e73-7b8d-408b-b356-5a6fcecb1981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:36.293073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9924e73-7b8d-408b-b356-5a6fcecb1981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:36.294534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9924e73-7b8d-408b-b356-5a6fcecb1981 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-a9924e73-7b8d-408b-b356-5a6fcecb1981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:36.312039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:36.313975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:36.315310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:36.315310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:36.315310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:36.315310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 618/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:36.319877 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecd6e8-7ffe-4745-a981-c0d87962a094 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:36.422306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:36.430380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecd6e8-7ffe-4745-a981-c0d87962a094 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:36.470482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87217c2-9b37-497c-9800-a758b7184388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:36.471115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87217c2-9b37-497c-9800-a758b7184388 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:36.471453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87217c2-9b37-497c-9800-a758b7184388 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:36.471755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87217c2-9b37-497c-9800-a758b7184388 None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:36.473028 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87217c2-9b37-497c-9800-a758b7184388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:36.473358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87217c2-9b37-497c-9800-a758b7184388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:36.474736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87217c2-9b37-497c-9800-a758b7184388 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:36.488517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 625/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:36.495077 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9924e73-7b8d-408b-b356-5a6fcecb1981 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:36.584835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:36.608612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9924e73-7b8d-408b-b356-5a6fcecb1981 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:36.649311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4cf5a-a23f-45ae-b4d3-ff395ee27bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:36.649977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4cf5a-a23f-45ae-b4d3-ff395ee27bd3 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:36.650333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4cf5a-a23f-45ae-b4d3-ff395ee27bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:36.650618 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4cf5a-a23f-45ae-b4d3-ff395ee27bd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:36.658154 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4cf5a-a23f-45ae-b4d3-ff395ee27bd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:36.660278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4cf5a-a23f-45ae-b4d3-ff395ee27bd3 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-bee4cf5a-a23f-45ae-b4d3-ff395ee27bd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:38:36.672058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:36.673238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:36.674153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:38:36.674153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:38:36.674153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:38:36.674153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:38:36.674153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:38:36.674153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:38:36.674153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:36.674153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:36.674153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 619/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:38:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:36.679906 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87217c2-9b37-497c-9800-a758b7184388 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:36.792705 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87217c2-9b37-497c-9800-a758b7184388 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:36.832102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91579da1-0fde-4b6e-a183-40a7eae7af0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:36.832934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91579da1-0fde-4b6e-a183-40a7eae7af0f None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:36.833294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91579da1-0fde-4b6e-a183-40a7eae7af0f None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:36.833609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91579da1-0fde-4b6e-a183-40a7eae7af0f None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:36.834602 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91579da1-0fde-4b6e-a183-40a7eae7af0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:36.834976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91579da1-0fde-4b6e-a183-40a7eae7af0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:36.836837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91579da1-0fde-4b6e-a183-40a7eae7af0f None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:36.853116 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 626/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:38:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:36.860597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4cf5a-a23f-45ae-b4d3-ff395ee27bd3 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:36.994774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4cf5a-a23f-45ae-b4d3-ff395ee27bd3 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:37.039099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c89eed-6436-4059-989c-5c30cd1beaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:37.039770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c89eed-6436-4059-989c-5c30cd1beaa0 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:37.040170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c89eed-6436-4059-989c-5c30cd1beaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:37.040412 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c89eed-6436-4059-989c-5c30cd1beaa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:37.041173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c89eed-6436-4059-989c-5c30cd1beaa0 None admin] RBAC: Adding query filter params (role.id=dcba9d07b31a4387b3014365e6e2ea10, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=560df3eb05f14b6489158ab053f282ca) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:37.041546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c89eed-6436-4059-989c-5c30cd1beaa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:37.043332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c89eed-6436-4059-989c-5c30cd1beaa0 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:37.051420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 620/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:38:36 2022] GET /identity/v3/role_assignments?user.id=560df3eb05f14b6489158ab053f282ca&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=dcba9d07b31a4387b3014365e6e2ea10 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:39.163209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36049d-38d6-433c-8ee7-541306ebc6b2 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:39.163831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36049d-38d6-433c-8ee7-541306ebc6b2 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:39.163831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36049d-38d6-433c-8ee7-541306ebc6b2 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:39.165246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 627/1247] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:39.170876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d646c4c-7778-4613-875e-ad5223fa1485 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:39.171494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d646c4c-7778-4613-875e-ad5223fa1485 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:39.171494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d646c4c-7778-4613-875e-ad5223fa1485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:39.179486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:39.193611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4d646c4c-7778-4613-875e-ad5223fa1485 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:39.194976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4d646c4c-7778-4613-875e-ad5223fa1485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:39.278045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 621/1248] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:39.592579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36049d-38d6-433c-8ee7-541306ebc6b2 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:39.613269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:39.712493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36049d-38d6-433c-8ee7-541306ebc6b2 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:39.757111 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10c3b4-1bf0-476c-af58-c9099b4011cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:39.757770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10c3b4-1bf0-476c-af58-c9099b4011cd None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:39.758087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10c3b4-1bf0-476c-af58-c9099b4011cd None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:39.758399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10c3b4-1bf0-476c-af58-c9099b4011cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:39.770564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10c3b4-1bf0-476c-af58-c9099b4011cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-9a10c3b4-1bf0-476c-af58-c9099b4011cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:38:39.777291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:39.778964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:39.780139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 628/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:38:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:39.786396 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d646c4c-7778-4613-875e-ad5223fa1485 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:39.903005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d646c4c-7778-4613-875e-ad5223fa1485 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:39.939043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:39.945935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095847b1-6da8-4914-982d-975ee58270e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:39.946622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095847b1-6da8-4914-982d-975ee58270e6 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:39.946906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095847b1-6da8-4914-982d-975ee58270e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:39.947239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095847b1-6da8-4914-982d-975ee58270e6 None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:39.947972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095847b1-6da8-4914-982d-975ee58270e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:39.948318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095847b1-6da8-4914-982d-975ee58270e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:39.949745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095847b1-6da8-4914-982d-975ee58270e6 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:39.960101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 622/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:38:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:39.966812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10c3b4-1bf0-476c-af58-c9099b4011cd None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:40.070469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:40.100363 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10c3b4-1bf0-476c-af58-c9099b4011cd None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:40.141189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213b202-7392-43c6-963c-8e382e8fe404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:40.141836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213b202-7392-43c6-963c-8e382e8fe404 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:40.142120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213b202-7392-43c6-963c-8e382e8fe404 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:40.142392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213b202-7392-43c6-963c-8e382e8fe404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:40.153811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213b202-7392-43c6-963c-8e382e8fe404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:40.155241 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213b202-7392-43c6-963c-8e382e8fe404 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-5213b202-7392-43c6-963c-8e382e8fe404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:40.171611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:40.173191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:40.174084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:40.174084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:40.174084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:40.174084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 629/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:40.179623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095847b1-6da8-4914-982d-975ee58270e6 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:40.307997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095847b1-6da8-4914-982d-975ee58270e6 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:40.350802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b0fb6-fa42-47c5-86dd-98ea945419c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:40.351498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b0fb6-fa42-47c5-86dd-98ea945419c4 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:40.351799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b0fb6-fa42-47c5-86dd-98ea945419c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:40.352057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b0fb6-fa42-47c5-86dd-98ea945419c4 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:40.352701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b0fb6-fa42-47c5-86dd-98ea945419c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:40.352999 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b0fb6-fa42-47c5-86dd-98ea945419c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:40.354433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b0fb6-fa42-47c5-86dd-98ea945419c4 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:40.370821 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 623/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:40.377789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213b202-7392-43c6-963c-8e382e8fe404 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:40.490589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213b202-7392-43c6-963c-8e382e8fe404 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:40.522325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:40.532820 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0003231-1f95-4f91-b0ae-cf6a52f9b4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:40.532820 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0003231-1f95-4f91-b0ae-cf6a52f9b4db None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:40.532820 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0003231-1f95-4f91-b0ae-cf6a52f9b4db None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:40.533227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0003231-1f95-4f91-b0ae-cf6a52f9b4db None admin] PATH_INFO: `/v3/users/placement` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:40.542122 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0003231-1f95-4f91-b0ae-cf6a52f9b4db None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:40.544077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0003231-1f95-4f91-b0ae-cf6a52f9b4db None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-c0003231-1f95-4f91-b0ae-cf6a52f9b4db None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:38:40.557809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:40.559152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:40.560319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:40.560319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:40.560319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:38:40.560319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:38:40.560319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:38:40.560319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:38:40.560319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:38:40.560319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:38:40.560319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:40.560319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:40.560319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 630/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:40.565653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b0fb6-fa42-47c5-86dd-98ea945419c4 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:40.630881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:40.677443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b0fb6-fa42-47c5-86dd-98ea945419c4 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:40.715618 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438f996-5b9f-4e35-8cf8-9855c2cb0baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:40.716255 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438f996-5b9f-4e35-8cf8-9855c2cb0baa None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:40.716535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438f996-5b9f-4e35-8cf8-9855c2cb0baa None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:40.716802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438f996-5b9f-4e35-8cf8-9855c2cb0baa None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:40.717660 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438f996-5b9f-4e35-8cf8-9855c2cb0baa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:40.717969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438f996-5b9f-4e35-8cf8-9855c2cb0baa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:40.719461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438f996-5b9f-4e35-8cf8-9855c2cb0baa None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:40.739221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 624/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Aug 31 09:38:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:40.745570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0003231-1f95-4f91-b0ae-cf6a52f9b4db None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:40.866173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0003231-1f95-4f91-b0ae-cf6a52f9b4db None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:40.919508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba33843-a384-4fe0-8aa7-95413a813b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:40.920115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba33843-a384-4fe0-8aa7-95413a813b1c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:40.920431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba33843-a384-4fe0-8aa7-95413a813b1c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:40.920663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba33843-a384-4fe0-8aa7-95413a813b1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:41.002039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:41.018003 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba33843-a384-4fe0-8aa7-95413a813b1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:41.019642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba33843-a384-4fe0-8aa7-95413a813b1c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:41.145483 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 631/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:38:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:38:41.151641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438f996-5b9f-4e35-8cf8-9855c2cb0baa None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:41.274724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438f996-5b9f-4e35-8cf8-9855c2cb0baa None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:41.289760 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:41.313679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d28c6-ef1c-4a16-8f02-cd3468519b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:41.314308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d28c6-ef1c-4a16-8f02-cd3468519b76 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:41.314617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d28c6-ef1c-4a16-8f02-cd3468519b76 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:41.314889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d28c6-ef1c-4a16-8f02-cd3468519b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:41.326869 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d28c6-ef1c-4a16-8f02-cd3468519b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:41.328368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d28c6-ef1c-4a16-8f02-cd3468519b76 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-4b1d28c6-ef1c-4a16-8f02-cd3468519b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:41.344484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:41.345726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:41.346906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:41.346906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:41.346906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:41.350702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 625/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:38:41.352306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba33843-a384-4fe0-8aa7-95413a813b1c None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:41.452100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:41.466576 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba33843-a384-4fe0-8aa7-95413a813b1c None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:41.507640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3e358-b49a-45f6-98bc-9334797b1abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:41.508275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3e358-b49a-45f6-98bc-9334797b1abb None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:41.508529 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3e358-b49a-45f6-98bc-9334797b1abb None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:41.508802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3e358-b49a-45f6-98bc-9334797b1abb None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:41.509446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3e358-b49a-45f6-98bc-9334797b1abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:41.509756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3e358-b49a-45f6-98bc-9334797b1abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:41.511150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3e358-b49a-45f6-98bc-9334797b1abb None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:41.524170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 632/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:41.530502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d28c6-ef1c-4a16-8f02-cd3468519b76 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:41.655447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d28c6-ef1c-4a16-8f02-cd3468519b76 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:41.693093 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b571b9f-3fc6-43d0-b15f-8e9f3c73e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:41.693737 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b571b9f-3fc6-43d0-b15f-8e9f3c73e64a None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:41.694014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b571b9f-3fc6-43d0-b15f-8e9f3c73e64a None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:41.694286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b571b9f-3fc6-43d0-b15f-8e9f3c73e64a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:41.705053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b571b9f-3fc6-43d0-b15f-8e9f3c73e64a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:41.706680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b571b9f-3fc6-43d0-b15f-8e9f3c73e64a None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-2b571b9f-3fc6-43d0-b15f-8e9f3c73e64a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:38:41.719115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:41.720276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:41.721559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:38:41.721559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:38:41.721559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:38:41.721559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:38:41.721559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:38:41.721559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:38:41.721559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:41.721559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:41.721559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 626/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:38:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:38:41.727113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3e358-b49a-45f6-98bc-9334797b1abb None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:41.858863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3e358-b49a-45f6-98bc-9334797b1abb None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:41.899998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29bb78-d68d-4e05-8262-916ad3a49377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:41.900639 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29bb78-d68d-4e05-8262-916ad3a49377 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:41.901014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29bb78-d68d-4e05-8262-916ad3a49377 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:41.901341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29bb78-d68d-4e05-8262-916ad3a49377 None admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:41.901980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29bb78-d68d-4e05-8262-916ad3a49377 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:41.902338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29bb78-d68d-4e05-8262-916ad3a49377 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:41.903768 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29bb78-d68d-4e05-8262-916ad3a49377 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:41.916990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 633/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:38:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:41.924785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b571b9f-3fc6-43d0-b15f-8e9f3c73e64a None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:41.966664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:42.064830 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b571b9f-3fc6-43d0-b15f-8e9f3c73e64a None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:42.102256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79925876-4844-4846-a446-408986be1c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:42.102849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79925876-4844-4846-a446-408986be1c87 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:42.103142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79925876-4844-4846-a446-408986be1c87 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:42.103411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79925876-4844-4846-a446-408986be1c87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:42.104129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79925876-4844-4846-a446-408986be1c87 None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=560df3eb05f14b6489158ab053f282ca) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:42.104426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79925876-4844-4846-a446-408986be1c87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:42.105874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79925876-4844-4846-a446-408986be1c87 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:42.123257 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 627/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:38:41 2022] GET /identity/v3/role_assignments?user.id=560df3eb05f14b6489158ab053f282ca&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:44.157234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a862ab-04b6-424b-af2a-5ffa5c976c48 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:44.157795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a862ab-04b6-424b-af2a-5ffa5c976c48 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:44.157795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a862ab-04b6-424b-af2a-5ffa5c976c48 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:44.159115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 634/1261] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:44.164581 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe30bec-d22f-41ae-a7e1-3e802531af4d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:44.165058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe30bec-d22f-41ae-a7e1-3e802531af4d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:44.165149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe30bec-d22f-41ae-a7e1-3e802531af4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:44.188503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-8fe30bec-d22f-41ae-a7e1-3e802531af4d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:44.190345 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-8fe30bec-d22f-41ae-a7e1-3e802531af4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:44.268944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 628/1262] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:44.548485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a862ab-04b6-424b-af2a-5ffa5c976c48 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:44.562123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:44.652934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a862ab-04b6-424b-af2a-5ffa5c976c48 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:44.689863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a740212-a086-4661-b927-5a36a4cbcf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:44.690499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a740212-a086-4661-b927-5a36a4cbcf40 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:44.690840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a740212-a086-4661-b927-5a36a4cbcf40 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:44.691184 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a740212-a086-4661-b927-5a36a4cbcf40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:44.701669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a740212-a086-4661-b927-5a36a4cbcf40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-0a740212-a086-4661-b927-5a36a4cbcf40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:38:44.706884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:44.708005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:44.709080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 635/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:38:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:44.714184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe30bec-d22f-41ae-a7e1-3e802531af4d None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:44.733259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:44.814087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe30bec-d22f-41ae-a7e1-3e802531af4d None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:44.848791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95274d00-eac7-4dd9-ad27-b95dc6bbbf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:44.849404 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95274d00-eac7-4dd9-ad27-b95dc6bbbf6e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:44.849675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95274d00-eac7-4dd9-ad27-b95dc6bbbf6e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:44.849945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95274d00-eac7-4dd9-ad27-b95dc6bbbf6e None admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:44.850608 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95274d00-eac7-4dd9-ad27-b95dc6bbbf6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:44.850903 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95274d00-eac7-4dd9-ad27-b95dc6bbbf6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:44.852303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95274d00-eac7-4dd9-ad27-b95dc6bbbf6e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:44.860809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 629/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:38:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:44.867807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a740212-a086-4661-b927-5a36a4cbcf40 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:44.942371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:44.977145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a740212-a086-4661-b927-5a36a4cbcf40 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:45.014698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f8827-c0ed-42b1-9374-5b1e8518deb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:45.015367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f8827-c0ed-42b1-9374-5b1e8518deb8 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:45.015651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f8827-c0ed-42b1-9374-5b1e8518deb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:45.015926 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f8827-c0ed-42b1-9374-5b1e8518deb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:45.027265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f8827-c0ed-42b1-9374-5b1e8518deb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:45.028715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f8827-c0ed-42b1-9374-5b1e8518deb8 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-193f8827-c0ed-42b1-9374-5b1e8518deb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:45.044176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:45.045328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:45.046419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:45.046419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:45.046419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:45.046419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 636/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:45.051371 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95274d00-eac7-4dd9-ad27-b95dc6bbbf6e None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:45.137802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:45.163668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95274d00-eac7-4dd9-ad27-b95dc6bbbf6e None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:45.198107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f8ab3-1211-4a30-8c57-ebf1a4d3c32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:45.198790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f8ab3-1211-4a30-8c57-ebf1a4d3c32f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:45.198966 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f8ab3-1211-4a30-8c57-ebf1a4d3c32f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:45.199259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f8ab3-1211-4a30-8c57-ebf1a4d3c32f None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:45.199892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f8ab3-1211-4a30-8c57-ebf1a4d3c32f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:45.200187 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f8ab3-1211-4a30-8c57-ebf1a4d3c32f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:45.201529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f8ab3-1211-4a30-8c57-ebf1a4d3c32f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:45.213677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 630/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:45.219990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f8827-c0ed-42b1-9374-5b1e8518deb8 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:45.338825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f8827-c0ed-42b1-9374-5b1e8518deb8 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:45.358969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:45.374004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e5f72-63d1-419e-81d1-bc8d6bbc6dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:45.374628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e5f72-63d1-419e-81d1-bc8d6bbc6dc1 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:45.374903 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e5f72-63d1-419e-81d1-bc8d6bbc6dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:45.375205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e5f72-63d1-419e-81d1-bc8d6bbc6dc1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:45.383451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e5f72-63d1-419e-81d1-bc8d6bbc6dc1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:45.385021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e5f72-63d1-419e-81d1-bc8d6bbc6dc1 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-8c1e5f72-63d1-419e-81d1-bc8d6bbc6dc1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:38:45.397474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:45.402783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:45.404015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:45.404015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:45.404015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:38:45.404015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:38:45.404015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:38:45.404015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:38:45.404015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:38:45.404015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:38:45.404015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:45.404015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:45.404015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 637/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:45.404677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f8ab3-1211-4a30-8c57-ebf1a4d3c32f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:45.506386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f8ab3-1211-4a30-8c57-ebf1a4d3c32f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:45.540954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea3bae-ad79-4837-999d-16287562b009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:45.541555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea3bae-ad79-4837-999d-16287562b009 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:45.541835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea3bae-ad79-4837-999d-16287562b009 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:45.542106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea3bae-ad79-4837-999d-16287562b009 None admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:45.542829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea3bae-ad79-4837-999d-16287562b009 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:45.543157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea3bae-ad79-4837-999d-16287562b009 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:45.544600 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea3bae-ad79-4837-999d-16287562b009 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:45.555089 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:45.564443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 631/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Aug 31 09:38:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:45.574054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e5f72-63d1-419e-81d1-bc8d6bbc6dc1 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:45.690131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e5f72-63d1-419e-81d1-bc8d6bbc6dc1 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:45.724266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fda4fe-0500-494a-895d-d66990d98d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:45.724894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fda4fe-0500-494a-895d-d66990d98d8a None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:45.725173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fda4fe-0500-494a-895d-d66990d98d8a None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:45.725443 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fda4fe-0500-494a-895d-d66990d98d8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:45.735969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fda4fe-0500-494a-895d-d66990d98d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:45.738099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fda4fe-0500-494a-895d-d66990d98d8a None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-c8fda4fe-0500-494a-895d-d66990d98d8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:45.752152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:45.753810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:45.754728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:45.754728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:45.754728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:45.754728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 638/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:45.760176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea3bae-ad79-4837-999d-16287562b009 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:45.865923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea3bae-ad79-4837-999d-16287562b009 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:45.903145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5815143-ee0a-44ff-9fb6-ddb66eb22016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:45.903840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5815143-ee0a-44ff-9fb6-ddb66eb22016 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:45.904116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5815143-ee0a-44ff-9fb6-ddb66eb22016 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:45.904386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5815143-ee0a-44ff-9fb6-ddb66eb22016 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:45.905029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5815143-ee0a-44ff-9fb6-ddb66eb22016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:45.905335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5815143-ee0a-44ff-9fb6-ddb66eb22016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:45.906778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5815143-ee0a-44ff-9fb6-ddb66eb22016 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:45.919392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 632/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:45.927025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fda4fe-0500-494a-895d-d66990d98d8a None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:45.972061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:46.048070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fda4fe-0500-494a-895d-d66990d98d8a None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:46.083874 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66a5f3-3658-4c4f-b11f-07e00bce8b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:46.084504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66a5f3-3658-4c4f-b11f-07e00bce8b90 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:46.084778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66a5f3-3658-4c4f-b11f-07e00bce8b90 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:46.085047 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66a5f3-3658-4c4f-b11f-07e00bce8b90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:46.091987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66a5f3-3658-4c4f-b11f-07e00bce8b90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:46.093421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66a5f3-3658-4c4f-b11f-07e00bce8b90 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-8b66a5f3-3658-4c4f-b11f-07e00bce8b90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:38:46.104743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:46.105824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:46.106928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:38:46.106928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:38:46.106928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:38:46.106928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:38:46.106928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:38:46.106928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:38:46.106928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:46.106928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:46.106928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 639/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:46.111977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5815143-ee0a-44ff-9fb6-ddb66eb22016 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:46.172544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:46.217652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5815143-ee0a-44ff-9fb6-ddb66eb22016 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:46.264785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233e61f-5730-42d3-9cf5-47dcc88371d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:46.265433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233e61f-5730-42d3-9cf5-47dcc88371d2 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:46.265753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233e61f-5730-42d3-9cf5-47dcc88371d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:46.266130 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233e61f-5730-42d3-9cf5-47dcc88371d2 None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:46.267061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233e61f-5730-42d3-9cf5-47dcc88371d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:46.267434 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233e61f-5730-42d3-9cf5-47dcc88371d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:46.270271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233e61f-5730-42d3-9cf5-47dcc88371d2 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:46.293886 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 633/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:38:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:46.301507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66a5f3-3658-4c4f-b11f-07e00bce8b90 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:46.398674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:46.407788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66a5f3-3658-4c4f-b11f-07e00bce8b90 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:46.441414 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1fe54-afa7-4743-b4e2-67126395ca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:46.442059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1fe54-afa7-4743-b4e2-67126395ca00 None admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:46.442338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1fe54-afa7-4743-b4e2-67126395ca00 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:46.442612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1fe54-afa7-4743-b4e2-67126395ca00 None admin] PATH_INFO: `/v3/projects/8ef8b586382941408cd88ee654c42d32/users/560df3eb05f14b6489158ab053f282ca/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:46.444540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1fe54-afa7-4743-b4e2-67126395ca00 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef8b586382941408cd88ee654c42d32, user_id=560df3eb05f14b6489158ab053f282ca, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:46.445960 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1fe54-afa7-4743-b4e2-67126395ca00 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:46.458439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 640/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 09:38:46 2022] PUT /identity/v3/projects/8ef8b586382941408cd88ee654c42d32/users/560df3eb05f14b6489158ab053f282ca/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:38:48.434518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437baf07-9ed1-490b-bbe0-d4c73e77d239 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:48.435043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437baf07-9ed1-490b-bbe0-d4c73e77d239 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:48.435043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437baf07-9ed1-490b-bbe0-d4c73e77d239 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:48.436420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 634/1274] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:48.442548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300908f7-6811-4b66-b276-e08b1b2654c5 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:48.443022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300908f7-6811-4b66-b276-e08b1b2654c5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:48.443185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300908f7-6811-4b66-b276-e08b1b2654c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:48.462385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-300908f7-6811-4b66-b276-e08b1b2654c5 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:48.463640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-300908f7-6811-4b66-b276-e08b1b2654c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:48.531774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 641/1275] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:48.806471 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437baf07-9ed1-490b-bbe0-d4c73e77d239 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:48.913660 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437baf07-9ed1-490b-bbe0-d4c73e77d239 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:48.931826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:48.949847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e857a-7696-4a29-a4b7-3320badefda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:48.950482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e857a-7696-4a29-a4b7-3320badefda4 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:48.950760 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e857a-7696-4a29-a4b7-3320badefda4 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:48.951030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e857a-7696-4a29-a4b7-3320badefda4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:48.961873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e857a-7696-4a29-a4b7-3320badefda4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-e24e857a-7696-4a29-a4b7-3320badefda4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:38:48.966661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:48.968317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:48.970184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 635/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 09:38:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:48.974329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300908f7-6811-4b66-b276-e08b1b2654c5 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:49.057598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:49.077583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300908f7-6811-4b66-b276-e08b1b2654c5 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:49.116006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde27c5-5d49-4839-9799-27069cc53283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:49.116630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde27c5-5d49-4839-9799-27069cc53283 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:49.116909 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde27c5-5d49-4839-9799-27069cc53283 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:49.117180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde27c5-5d49-4839-9799-27069cc53283 None admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:49.117848 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde27c5-5d49-4839-9799-27069cc53283 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:49.118151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde27c5-5d49-4839-9799-27069cc53283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:49.119551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde27c5-5d49-4839-9799-27069cc53283 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:49.128753 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 642/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 09:38:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:49.134586 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e857a-7696-4a29-a4b7-3320badefda4 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:49.247552 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e857a-7696-4a29-a4b7-3320badefda4 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:49.289538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58274-2062-45c4-935e-05755ac9cf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:49.290301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58274-2062-45c4-935e-05755ac9cf65 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:49.290629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58274-2062-45c4-935e-05755ac9cf65 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:49.290999 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58274-2062-45c4-935e-05755ac9cf65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:49.305851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee58274-2062-45c4-935e-05755ac9cf65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:49.307718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee58274-2062-45c4-935e-05755ac9cf65 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-fee58274-2062-45c4-935e-05755ac9cf65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:49.330811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:49.332732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:49.333650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:49.333650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:49.333650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:49.333650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 636/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:49.340364 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde27c5-5d49-4839-9799-27069cc53283 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:49.447596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde27c5-5d49-4839-9799-27069cc53283 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:49.480983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42d32d-d121-412d-93fb-f2aa5507a5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:49.481580 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d32d-d121-412d-93fb-f2aa5507a5cd None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:49.481862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d32d-d121-412d-93fb-f2aa5507a5cd None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:49.482135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d32d-d121-412d-93fb-f2aa5507a5cd None admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:49.482955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42d32d-d121-412d-93fb-f2aa5507a5cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:49.483093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42d32d-d121-412d-93fb-f2aa5507a5cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:49.484486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42d32d-d121-412d-93fb-f2aa5507a5cd None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:49.496350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 643/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:49.502055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58274-2062-45c4-935e-05755ac9cf65 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:49.551615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:49.627022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58274-2062-45c4-935e-05755ac9cf65 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:49.662634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484216-cc6b-46f1-abda-b5fee8113fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:49.663280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484216-cc6b-46f1-abda-b5fee8113fe8 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:49.663556 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484216-cc6b-46f1-abda-b5fee8113fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:49.663820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484216-cc6b-46f1-abda-b5fee8113fe8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:49.671456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484216-cc6b-46f1-abda-b5fee8113fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:49.672892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484216-cc6b-46f1-abda-b5fee8113fe8 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-df484216-cc6b-46f1-abda-b5fee8113fe8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:38:49.683897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:49.685034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:49.685977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:49.685977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:49.685977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:38:49.685977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:38:49.685977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:38:49.685977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:38:49.685977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:38:49.685977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:38:49.685977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:49.685977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:49.685977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 637/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:49.691005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42d32d-d121-412d-93fb-f2aa5507a5cd None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:49.694917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:49.798478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42d32d-d121-412d-93fb-f2aa5507a5cd None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:49.837516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97d044-bc41-4a61-b99b-a0fa1c2e8d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:49.838132 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97d044-bc41-4a61-b99b-a0fa1c2e8d10 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:49.838411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97d044-bc41-4a61-b99b-a0fa1c2e8d10 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:49.838682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97d044-bc41-4a61-b99b-a0fa1c2e8d10 None admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:49.839456 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97d044-bc41-4a61-b99b-a0fa1c2e8d10 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:49.839761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97d044-bc41-4a61-b99b-a0fa1c2e8d10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:49.841198 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97d044-bc41-4a61-b99b-a0fa1c2e8d10 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:49.859904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 644/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Aug 31 09:38:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:49.866537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484216-cc6b-46f1-abda-b5fee8113fe8 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:49.959779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:49.969840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484216-cc6b-46f1-abda-b5fee8113fe8 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:50.002148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82053488-8415-4689-8790-cb4663dcfe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:50.003291 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82053488-8415-4689-8790-cb4663dcfe02 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:50.003566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82053488-8415-4689-8790-cb4663dcfe02 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:50.003831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82053488-8415-4689-8790-cb4663dcfe02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:50.074424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82053488-8415-4689-8790-cb4663dcfe02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:50.076579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82053488-8415-4689-8790-cb4663dcfe02 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:50.168345 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:50.183722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 638/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 09:38:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:38:50.189634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97d044-bc41-4a61-b99b-a0fa1c2e8d10 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:50.259774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:50.309607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97d044-bc41-4a61-b99b-a0fa1c2e8d10 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:50.345935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb490f2f-8c8c-4455-a7af-4fe9ccf89277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:50.346677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb490f2f-8c8c-4455-a7af-4fe9ccf89277 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:50.347157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb490f2f-8c8c-4455-a7af-4fe9ccf89277 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:50.347433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb490f2f-8c8c-4455-a7af-4fe9ccf89277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:50.358018 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb490f2f-8c8c-4455-a7af-4fe9ccf89277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:50.359502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb490f2f-8c8c-4455-a7af-4fe9ccf89277 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-bb490f2f-8c8c-4455-a7af-4fe9ccf89277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:50.378384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:50.380080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:50.381339 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:50.381339 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:50.381339 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:50.381339 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 645/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:50.385923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82053488-8415-4689-8790-cb4663dcfe02 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:50.497245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82053488-8415-4689-8790-cb4663dcfe02 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:50.531930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894eeb9-64c1-490a-99b5-a30741f170f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:50.532588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894eeb9-64c1-490a-99b5-a30741f170f4 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:50.532701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894eeb9-64c1-490a-99b5-a30741f170f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:50.532976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894eeb9-64c1-490a-99b5-a30741f170f4 None admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:50.533612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894eeb9-64c1-490a-99b5-a30741f170f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:50.533993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894eeb9-64c1-490a-99b5-a30741f170f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:50.536292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894eeb9-64c1-490a-99b5-a30741f170f4 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:50.547936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 639/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:50.554088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb490f2f-8c8c-4455-a7af-4fe9ccf89277 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:50.661350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb490f2f-8c8c-4455-a7af-4fe9ccf89277 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:50.677255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:50.699219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b68517-2bfc-4ba0-930c-385ebaeee1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:50.699847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b68517-2bfc-4ba0-930c-385ebaeee1a5 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:50.700127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b68517-2bfc-4ba0-930c-385ebaeee1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:50.700417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b68517-2bfc-4ba0-930c-385ebaeee1a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:50.708491 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b68517-2bfc-4ba0-930c-385ebaeee1a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:50.709955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b68517-2bfc-4ba0-930c-385ebaeee1a5 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application [None req-94b68517-2bfc-4ba0-930c-385ebaeee1a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:38:50.722434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:50.723596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:50.724677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:38:50.724677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:38:50.724677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:38:50.724677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:38:50.724677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:38:50.724677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:38:50.724677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:50.724677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: ERROR keystone.server.flask.application Aug 31 09:38:50.724677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 646/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 09:38:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:50.730451 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894eeb9-64c1-490a-99b5-a30741f170f4 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:50.782078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:50.835653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894eeb9-64c1-490a-99b5-a30741f170f4 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:50.875346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef3ac9-05b6-4cf1-9271-232bc18594d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:50.875970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef3ac9-05b6-4cf1-9271-232bc18594d8 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:50.876246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef3ac9-05b6-4cf1-9271-232bc18594d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:50.876516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef3ac9-05b6-4cf1-9271-232bc18594d8 None admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:50.877229 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef3ac9-05b6-4cf1-9271-232bc18594d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:50.877533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef3ac9-05b6-4cf1-9271-232bc18594d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:50.879023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef3ac9-05b6-4cf1-9271-232bc18594d8 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:50.892100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 640/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 09:38:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:50.898413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b68517-2bfc-4ba0-930c-385ebaeee1a5 None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:50.997084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b68517-2bfc-4ba0-930c-385ebaeee1a5 None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:51.030897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91617617-b0dd-4294-ae56-b5c257daedfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:51.031513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91617617-b0dd-4294-ae56-b5c257daedfa None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:51.031788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91617617-b0dd-4294-ae56-b5c257daedfa None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:51.032054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91617617-b0dd-4294-ae56-b5c257daedfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:51.032815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91617617-b0dd-4294-ae56-b5c257daedfa None admin] RBAC: Adding query filter params (role.id=e7c31e56309e4f3880ee08fc63752e5d, scope.project.id=8ef8b586382941408cd88ee654c42d32, user.id=560df3eb05f14b6489158ab053f282ca) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:51.033063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91617617-b0dd-4294-ae56-b5c257daedfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:51.034735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91617617-b0dd-4294-ae56-b5c257daedfa None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:51.040538 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 647/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 09:38:50 2022] GET /identity/v3/role_assignments?user.id=560df3eb05f14b6489158ab053f282ca&scope.project.id=8ef8b586382941408cd88ee654c42d32&role.id=e7c31e56309e4f3880ee08fc63752e5d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:52.999958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baecf95-caf8-41dd-a1fa-24cabcb1bba8 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:53.000524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baecf95-caf8-41dd-a1fa-24cabcb1bba8 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:53.000524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baecf95-caf8-41dd-a1fa-24cabcb1bba8 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:53.001796 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 641/1288] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:53.007622 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f82fc-a426-457a-b0f0-991ae8ba8bf1 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:53.008049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f82fc-a426-457a-b0f0-991ae8ba8bf1 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:53.008156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f82fc-a426-457a-b0f0-991ae8ba8bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:53.026869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-c51f82fc-a426-457a-b0f0-991ae8ba8bf1 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:53.028089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c51f82fc-a426-457a-b0f0-991ae8ba8bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:53.056799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:53.095485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 648/1289] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:53.439924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baecf95-caf8-41dd-a1fa-24cabcb1bba8 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:53.541542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baecf95-caf8-41dd-a1fa-24cabcb1bba8 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:53.552289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:53.576446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038e0f5-59df-4071-a816-a0a32da9855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:53.577071 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038e0f5-59df-4071-a816-a0a32da9855f None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:53.577350 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038e0f5-59df-4071-a816-a0a32da9855f None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:53.577621 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038e0f5-59df-4071-a816-a0a32da9855f None admin] PATH_INFO: `/v3/services/placement` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:53.578193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038e0f5-59df-4071-a816-a0a32da9855f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:53.579597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038e0f5-59df-4071-a816-a0a32da9855f None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-1038e0f5-59df-4071-a816-a0a32da9855f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:53.588856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:53.590107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:53.591189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 642/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 09:38:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:53.596154 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f82fc-a426-457a-b0f0-991ae8ba8bf1 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:53.692973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f82fc-a426-457a-b0f0-991ae8ba8bf1 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:53.726693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f208153-fa3c-43bd-afc4-ddbfb52a8f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:53.727318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f208153-fa3c-43bd-afc4-ddbfb52a8f9c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:53.727601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f208153-fa3c-43bd-afc4-ddbfb52a8f9c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:53.727873 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f208153-fa3c-43bd-afc4-ddbfb52a8f9c None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:53.728498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f208153-fa3c-43bd-afc4-ddbfb52a8f9c None admin] RBAC: Adding query filter params (name=placement) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:53.728798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f208153-fa3c-43bd-afc4-ddbfb52a8f9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:53.730158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f208153-fa3c-43bd-afc4-ddbfb52a8f9c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:53.734900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 649/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Aug 31 09:38:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:53.740859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038e0f5-59df-4071-a816-a0a32da9855f None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:53.840895 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038e0f5-59df-4071-a816-a0a32da9855f None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:53.873761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e5614-f20e-4daf-b559-4020722640ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:53.874358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5614-f20e-4daf-b559-4020722640ef None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:53.874629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5614-f20e-4daf-b559-4020722640ef None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:53.874894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5614-f20e-4daf-b559-4020722640ef None admin] PATH_INFO: `/v3/services` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:53.875533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e5614-f20e-4daf-b559-4020722640ef None admin] RBAC: Adding query filter params (type=placement) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:53.875829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e5614-f20e-4daf-b559-4020722640ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:53.877196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e5614-f20e-4daf-b559-4020722640ef None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:53.882052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 643/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Aug 31 09:38:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:55.798204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491daf9-c1f7-49ed-be0e-a184f399a605 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:55.798732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491daf9-c1f7-49ed-be0e-a184f399a605 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:55.798732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491daf9-c1f7-49ed-be0e-a184f399a605 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:55.800246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 650/1293] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:55.805840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f68fcc-7bf6-459e-9eee-151aaa3769d8 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:55.806243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f68fcc-7bf6-459e-9eee-151aaa3769d8 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:55.806367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f68fcc-7bf6-459e-9eee-151aaa3769d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:55.824055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f9f68fcc-7bf6-459e-9eee-151aaa3769d8 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:55.825219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f9f68fcc-7bf6-459e-9eee-151aaa3769d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:55.854732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:55.893448 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 644/1294] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:56.160971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491daf9-c1f7-49ed-be0e-a184f399a605 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:56.191515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:56.261214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491daf9-c1f7-49ed-be0e-a184f399a605 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:56.304502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfb220-5771-4bed-beb8-a2d57de9c282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:56.305292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfb220-5771-4bed-beb8-a2d57de9c282 None admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:56.305658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfb220-5771-4bed-beb8-a2d57de9c282 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:56.306115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfb220-5771-4bed-beb8-a2d57de9c282 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:56.307404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfb220-5771-4bed-beb8-a2d57de9c282 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:56.309455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfb220-5771-4bed-beb8-a2d57de9c282 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:56.327851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 651/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 09:38:56 2022] POST /identity/v3/services => generated 246 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:38:58.292683 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01842783-fcb0-45e7-83cb-dbe100d40728 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:58.293260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01842783-fcb0-45e7-83cb-dbe100d40728 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:58.293260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01842783-fcb0-45e7-83cb-dbe100d40728 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:58.294568 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 645/1296] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:58.299908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bac6e8-d4de-4582-a994-a25f71a70f0d None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:58.300328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bac6e8-d4de-4582-a994-a25f71a70f0d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:58.300454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bac6e8-d4de-4582-a994-a25f71a70f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:58.318503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-55bac6e8-d4de-4582-a994-a25f71a70f0d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:58.319699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-55bac6e8-d4de-4582-a994-a25f71a70f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:58.394230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 652/1297] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:58 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:58.662636 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01842783-fcb0-45e7-83cb-dbe100d40728 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:58.766650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01842783-fcb0-45e7-83cb-dbe100d40728 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:58.802955 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a4357-ed63-4975-b3ba-a87a42ba133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:58.803608 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a4357-ed63-4975-b3ba-a87a42ba133d None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:58.803885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a4357-ed63-4975-b3ba-a87a42ba133d None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:58.804155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a4357-ed63-4975-b3ba-a87a42ba133d None admin] PATH_INFO: `/v3/services/placement` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:58.804731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a4357-ed63-4975-b3ba-a87a42ba133d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:58.806098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a4357-ed63-4975-b3ba-a87a42ba133d None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-f00a4357-ed63-4975-b3ba-a87a42ba133d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:58.814972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:58.816609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:38:58.817437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 646/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 09:38:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:58.822271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bac6e8-d4de-4582-a994-a25f71a70f0d None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:58.843971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:58.927858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bac6e8-d4de-4582-a994-a25f71a70f0d None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:58.962796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec21f9f-4cb3-4521-bd72-65bebaa5e3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:58.963433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec21f9f-4cb3-4521-bd72-65bebaa5e3bd None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:58.963714 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec21f9f-4cb3-4521-bd72-65bebaa5e3bd None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:58.963984 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec21f9f-4cb3-4521-bd72-65bebaa5e3bd None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:58.964612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec21f9f-4cb3-4521-bd72-65bebaa5e3bd None admin] RBAC: Adding query filter params (name=placement) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:58.964915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec21f9f-4cb3-4521-bd72-65bebaa5e3bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:58.966291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec21f9f-4cb3-4521-bd72-65bebaa5e3bd None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:58.972793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 653/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Aug 31 09:38:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:58.979290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a4357-ed63-4975-b3ba-a87a42ba133d None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:59.010531 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:59.083185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a4357-ed63-4975-b3ba-a87a42ba133d None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:59.117532 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3863c5-0d26-4ccd-b492-9edd6f687a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:59.118133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3863c5-0d26-4ccd-b492-9edd6f687a2e None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:59.118409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3863c5-0d26-4ccd-b492-9edd6f687a2e None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:59.118678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3863c5-0d26-4ccd-b492-9edd6f687a2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:59.119400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3863c5-0d26-4ccd-b492-9edd6f687a2e None admin] RBAC: Adding query filter params (interface=public, service_id=dd0845b6d91c4976b2225f803e02a405, region_id=RegionOne) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:59.119705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3863c5-0d26-4ccd-b492-9edd6f687a2e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:59.121067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3863c5-0d26-4ccd-b492-9edd6f687a2e None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:59.125974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 647/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 09:38:58 2022] GET /identity/v3/endpoints?service_id=dd0845b6d91c4976b2225f803e02a405&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:59.132670 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec21f9f-4cb3-4521-bd72-65bebaa5e3bd None admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:59.217796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:59.235781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec21f9f-4cb3-4521-bd72-65bebaa5e3bd None admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:59.274752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e92a0-f24b-4e2a-9104-acdc52683c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:59.275475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e92a0-f24b-4e2a-9104-acdc52683c3c None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:59.275757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e92a0-f24b-4e2a-9104-acdc52683c3c None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:59.276155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e92a0-f24b-4e2a-9104-acdc52683c3c None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:59.276964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e92a0-f24b-4e2a-9104-acdc52683c3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:59.278767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e92a0-f24b-4e2a-9104-acdc52683c3c None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:59.287175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 654/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 09:38:59 2022] GET /identity/v3/services => generated 2191 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:39:01.220709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58702f7-2661-4179-b989-af96608d4e4e None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:01.221407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58702f7-2661-4179-b989-af96608d4e4e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:01.221865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58702f7-2661-4179-b989-af96608d4e4e None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:01.223547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 648/1302] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:39:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:01.228654 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a7a80-05f0-4d56-bbdf-ee9c3c141aec None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:01.229141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a7a80-05f0-4d56-bbdf-ee9c3c141aec None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:01.229236 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a7a80-05f0-4d56-bbdf-ee9c3c141aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:01.247798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-390a7a80-05f0-4d56-bbdf-ee9c3c141aec None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:01.248998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-390a7a80-05f0-4d56-bbdf-ee9c3c141aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:01.325276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 655/1303] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:39:01.596266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58702f7-2661-4179-b989-af96608d4e4e None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:01.693113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:01.700173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58702f7-2661-4179-b989-af96608d4e4e None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:01.733695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a724aaf-c41a-4dbe-90cb-a39eb1cb7c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:01.734308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a724aaf-c41a-4dbe-90cb-a39eb1cb7c81 None admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:01.734583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a724aaf-c41a-4dbe-90cb-a39eb1cb7c81 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:01.734851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a724aaf-c41a-4dbe-90cb-a39eb1cb7c81 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:01.735447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a724aaf-c41a-4dbe-90cb-a39eb1cb7c81 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:01.736840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a724aaf-c41a-4dbe-90cb-a39eb1cb7c81 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application [None req-1a724aaf-c41a-4dbe-90cb-a39eb1cb7c81 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:39:01.745411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:39:01.746619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application with Lock( Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: ERROR keystone.server.flask.application Aug 31 09:39:01.747542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 649/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 09:39:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:39:01.752752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a7a80-05f0-4d56-bbdf-ee9c3c141aec None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:01.827861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:01.853119 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a7a80-05f0-4d56-bbdf-ee9c3c141aec None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:01.886270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce5f33-caac-40bf-bbe3-62e2906c70a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:01.886860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce5f33-caac-40bf-bbe3-62e2906c70a6 None admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:01.887160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce5f33-caac-40bf-bbe3-62e2906c70a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:01.887432 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce5f33-caac-40bf-bbe3-62e2906c70a6 None admin] PATH_INFO: `/v3/services` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:01.888053 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce5f33-caac-40bf-bbe3-62e2906c70a6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:39:01.888349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce5f33-caac-40bf-bbe3-62e2906c70a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:01.889694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce5f33-caac-40bf-bbe3-62e2906c70a6 None admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:01.894606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 656/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Aug 31 09:39:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:39:01.900361 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a724aaf-c41a-4dbe-90cb-a39eb1cb7c81 None admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:01.999524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a724aaf-c41a-4dbe-90cb-a39eb1cb7c81 None admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:02.031749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2db6d-e51b-48cb-904b-ab039a3a1692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:02.032346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2db6d-e51b-48cb-904b-ab039a3a1692 None admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:02.032620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2db6d-e51b-48cb-904b-ab039a3a1692 None admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:02.032890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2db6d-e51b-48cb-904b-ab039a3a1692 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:02.034082 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2db6d-e51b-48cb-904b-ab039a3a1692 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:02.035487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2db6d-e51b-48cb-904b-ab039a3a1692 None admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:02.054981 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 650/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:39:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:39:11.892269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=30 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:12.050451 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:10.002315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf4218-2372-4bfb-8880-e7e25580747b None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:10.003434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf4218-2372-4bfb-8880-e7e25580747b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:10.003933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf4218-2372-4bfb-8880-e7e25580747b None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:10.005894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 657/1307] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:10.012121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf69c57-10da-471d-b774-8a531ff65826 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:10.012644 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf69c57-10da-471d-b774-8a531ff65826 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:10.012734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf69c57-10da-471d-b774-8a531ff65826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:10.136595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-3bf69c57-10da-471d-b774-8a531ff65826 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:10.137983 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3bf69c57-10da-471d-b774-8a531ff65826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:10.276957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 651/1308] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:10.586514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828e5a4-f6ea-49c6-9af6-89138a5a8547 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:10.586871 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828e5a4-f6ea-49c6-9af6-89138a5a8547 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:10.587426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828e5a4-f6ea-49c6-9af6-89138a5a8547 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:10.588931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 658/1309] 10.222.0.36 () {54 vars in 965 bytes} [Wed Aug 31 09:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:10.597051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09012ac7-8fb7-4953-937a-808dc01adf8f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:10.597339 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09012ac7-8fb7-4953-937a-808dc01adf8f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:10.597819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09012ac7-8fb7-4953-937a-808dc01adf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:10.701218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-09012ac7-8fb7-4953-937a-808dc01adf8f None None] MFA Rules not processed for user `6ff1177119cb48e0a88b0f11a6b1b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:10.703314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-09012ac7-8fb7-4953-937a-808dc01adf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:10.752786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:10.829588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 652/1310] 10.222.0.36 () {58 vars in 1062 bytes} [Wed Aug 31 09:40:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:10.841345 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828e5a4-f6ea-49c6-9af6-89138a5a8547 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:10.844418 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6828e5a4-f6ea-49c6-9af6-89138a5a8547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:10.987565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828e5a4-f6ea-49c6-9af6-89138a5a8547 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:11.106471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34f69d-373c-4732-9d6b-59ae1ebc5488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:11.107397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34f69d-373c-4732-9d6b-59ae1ebc5488 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:11.107830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34f69d-373c-4732-9d6b-59ae1ebc5488 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:11.108409 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34f69d-373c-4732-9d6b-59ae1ebc5488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:11.111444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1a34f69d-373c-4732-9d6b-59ae1ebc5488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:11.195288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:11.274951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34f69d-373c-4732-9d6b-59ae1ebc5488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:11.276448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34f69d-373c-4732-9d6b-59ae1ebc5488 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:11.299671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 659/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:40:13.666021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4e3d7-26a8-46f1-9d12-fd9a411737f9 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:13.666597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4e3d7-26a8-46f1-9d12-fd9a411737f9 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:13.666597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4e3d7-26a8-46f1-9d12-fd9a411737f9 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:13.667830 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 653/1312] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:13.674601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7463c64-570b-4249-9e18-c47761ee827f None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:13.675181 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7463c64-570b-4249-9e18-c47761ee827f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:13.675771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7463c64-570b-4249-9e18-c47761ee827f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:13.776224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-c7463c64-570b-4249-9e18-c47761ee827f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:13.777475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c7463c64-570b-4249-9e18-c47761ee827f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:13.896438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 660/1313] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:14.193620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4e3d7-26a8-46f1-9d12-fd9a411737f9 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:14.195166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f8c4e3d7-26a8-46f1-9d12-fd9a411737f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:14.329433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4e3d7-26a8-46f1-9d12-fd9a411737f9 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:14.424779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:14.429041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99b6b1-cd44-4803-9768-31143463cb93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:14.430013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99b6b1-cd44-4803-9768-31143463cb93 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:14.430657 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99b6b1-cd44-4803-9768-31143463cb93 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:14.431105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99b6b1-cd44-4803-9768-31143463cb93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:14.434455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4f99b6b1-cd44-4803-9768-31143463cb93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:14.559621 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99b6b1-cd44-4803-9768-31143463cb93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:14.560963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99b6b1-cd44-4803-9768-31143463cb93 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:14.579987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 654/1314] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:16.896744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd5a3a-feb2-4aba-86e5-ca1d4922dc1b None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:16.897359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd5a3a-feb2-4aba-86e5-ca1d4922dc1b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:16.897359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd5a3a-feb2-4aba-86e5-ca1d4922dc1b None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:16.898875 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 661/1315] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:16.904815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2a505-9bd8-4cc8-85ca-859a4109931e None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:16.906084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2a505-9bd8-4cc8-85ca-859a4109931e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:16.906084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2a505-9bd8-4cc8-85ca-859a4109931e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:16.989887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-10c2a505-9bd8-4cc8-85ca-859a4109931e None None] MFA Rules not processed for user `dd9b2ddbfdf942fbab0323fe249d8639`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:16.991637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-10c2a505-9bd8-4cc8-85ca-859a4109931e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:17.054871 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:17.108044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 655/1316] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:40:17.422083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bec371-63bc-447a-9281-fd6a0f293051 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:17.422368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bec371-63bc-447a-9281-fd6a0f293051 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:17.422634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bec371-63bc-447a-9281-fd6a0f293051 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:17.424188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 662/1317] 10.222.0.36 () {54 vars in 965 bytes} [Wed Aug 31 09:40:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:17.430724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8d89e-653a-4f34-a6fe-7c0d2a5811da None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:17.431057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8d89e-653a-4f34-a6fe-7c0d2a5811da None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:17.431408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8d89e-653a-4f34-a6fe-7c0d2a5811da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:17.514099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4fe8d89e-653a-4f34-a6fe-7c0d2a5811da None None] MFA Rules not processed for user `6ff1177119cb48e0a88b0f11a6b1b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:17.516078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4fe8d89e-653a-4f34-a6fe-7c0d2a5811da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:17.600408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 656/1318] 10.222.0.36 () {58 vars in 1062 bytes} [Wed Aug 31 09:40:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:17.610175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bec371-63bc-447a-9281-fd6a0f293051 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:17.612791 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-58bec371-63bc-447a-9281-fd6a0f293051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:17.629953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:17.772928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bec371-63bc-447a-9281-fd6a0f293051 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:17.865079 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9780d1-b6eb-4de2-9497-f58d00ffdd89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:17.865604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9780d1-b6eb-4de2-9497-f58d00ffdd89 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:17.865884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9780d1-b6eb-4de2-9497-f58d00ffdd89 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:17.866154 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9780d1-b6eb-4de2-9497-f58d00ffdd89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:17.867826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6d9780d1-b6eb-4de2-9497-f58d00ffdd89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:17.937746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:17.990573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9780d1-b6eb-4de2-9497-f58d00ffdd89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:17.992046 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9780d1-b6eb-4de2-9497-f58d00ffdd89 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:18.012115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 663/1319] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:17 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:20.998433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67157a36-b454-4eef-81b3-8c2ee170e4b3 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:20.998982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67157a36-b454-4eef-81b3-8c2ee170e4b3 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:20.998982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67157a36-b454-4eef-81b3-8c2ee170e4b3 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:21.000942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 657/1320] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:21.006559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842536f-9aae-4f75-a09c-32ef6048baf2 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:21.007168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842536f-9aae-4f75-a09c-32ef6048baf2 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:21.007168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842536f-9aae-4f75-a09c-32ef6048baf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:21.082460 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b842536f-9aae-4f75-a09c-32ef6048baf2 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:21.083769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b842536f-9aae-4f75-a09c-32ef6048baf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:21.188907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 664/1321] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:21.479823 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67157a36-b454-4eef-81b3-8c2ee170e4b3 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:21.481347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-67157a36-b454-4eef-81b3-8c2ee170e4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:21.537297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:21.597926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67157a36-b454-4eef-81b3-8c2ee170e4b3 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:21.676310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19453bce-2675-47fd-907e-7bfc41839a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:21.676917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19453bce-2675-47fd-907e-7bfc41839a5f service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:21.677190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19453bce-2675-47fd-907e-7bfc41839a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:21.677458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19453bce-2675-47fd-907e-7bfc41839a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:21.679073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-19453bce-2675-47fd-907e-7bfc41839a5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:21.779979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:21.794255 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19453bce-2675-47fd-907e-7bfc41839a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:21.795754 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19453bce-2675-47fd-907e-7bfc41839a5f service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:21.817674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 658/1322] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:24.734323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c412b6-8a99-4887-a0fb-b694f4c6a9e0 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:24.734889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c412b6-8a99-4887-a0fb-b694f4c6a9e0 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:24.734889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c412b6-8a99-4887-a0fb-b694f4c6a9e0 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:24.736449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 665/1323] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:24.742327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87380c34-9c58-44ab-85f8-4c6704c08264 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:24.742882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87380c34-9c58-44ab-85f8-4c6704c08264 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:24.743020 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87380c34-9c58-44ab-85f8-4c6704c08264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:24.822384 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-87380c34-9c58-44ab-85f8-4c6704c08264 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:24.823673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-87380c34-9c58-44ab-85f8-4c6704c08264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:24.935084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 659/1324] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:25.227181 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c412b6-8a99-4887-a0fb-b694f4c6a9e0 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:25.228837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-87c412b6-8a99-4887-a0fb-b694f4c6a9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:25.254717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:25.356929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c412b6-8a99-4887-a0fb-b694f4c6a9e0 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:25.446986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d33dd2-d4f1-41a0-8836-324873c1fa9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:25.447639 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d33dd2-d4f1-41a0-8836-324873c1fa9d service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:25.447923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d33dd2-d4f1-41a0-8836-324873c1fa9d service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:25.448169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d33dd2-d4f1-41a0-8836-324873c1fa9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:25.449929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a7d33dd2-d4f1-41a0-8836-324873c1fa9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:25.538938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:25.582375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d33dd2-d4f1-41a0-8836-324873c1fa9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:25.583881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d33dd2-d4f1-41a0-8836-324873c1fa9d service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:25.606429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 666/1325] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:27.700501 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ea1d5-6083-44ba-bdae-a7063c3d4f59 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:27.701292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ea1d5-6083-44ba-bdae-a7063c3d4f59 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:27.701292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ea1d5-6083-44ba-bdae-a7063c3d4f59 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:27.702959 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 660/1326] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:40:27.710076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896de61c-35a0-48f6-9315-8d589748ca8b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:27.710684 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896de61c-35a0-48f6-9315-8d589748ca8b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:27.711063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896de61c-35a0-48f6-9315-8d589748ca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:27.789872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-896de61c-35a0-48f6-9315-8d589748ca8b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:27.791143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-896de61c-35a0-48f6-9315-8d589748ca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:27.908430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 667/1327] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:28.196859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ea1d5-6083-44ba-bdae-a7063c3d4f59 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:28.199806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3d3ea1d5-6083-44ba-bdae-a7063c3d4f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:28.295592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:28.346861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ea1d5-6083-44ba-bdae-a7063c3d4f59 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:28.442491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2e0cd-ef97-4bf6-ae3d-3fd67d8a0e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:28.443446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e0cd-ef97-4bf6-ae3d-3fd67d8a0e5d service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:28.443869 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e0cd-ef97-4bf6-ae3d-3fd67d8a0e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:28.444302 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e0cd-ef97-4bf6-ae3d-3fd67d8a0e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:28.447547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-90d2e0cd-ef97-4bf6-ae3d-3fd67d8a0e5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:28.613160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:28.616822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2e0cd-ef97-4bf6-ae3d-3fd67d8a0e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:28.619190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2e0cd-ef97-4bf6-ae3d-3fd67d8a0e5d service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:28.641997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 661/1328] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:30.695979 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76634ed9-8e5c-4e54-a2d5-631df45e2e28 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:30.696763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76634ed9-8e5c-4e54-a2d5-631df45e2e28 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:30.696763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76634ed9-8e5c-4e54-a2d5-631df45e2e28 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:30.697986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 668/1329] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:30.704806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0e669-c76f-4959-bbe7-6a5cfecbd46f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:30.705392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0e669-c76f-4959-bbe7-6a5cfecbd46f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:30.705392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0e669-c76f-4959-bbe7-6a5cfecbd46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:30.786444 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-24b0e669-c76f-4959-bbe7-6a5cfecbd46f None None] MFA Rules not processed for user `dd9b2ddbfdf942fbab0323fe249d8639`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:30.787675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-24b0e669-c76f-4959-bbe7-6a5cfecbd46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:30.909314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 662/1330] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:31.207283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76634ed9-8e5c-4e54-a2d5-631df45e2e28 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:31.209191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-76634ed9-8e5c-4e54-a2d5-631df45e2e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:31.253350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:31.340804 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76634ed9-8e5c-4e54-a2d5-631df45e2e28 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:31.466749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008279f-780b-4cc9-a695-d0d2c429c8cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:31.467426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008279f-780b-4cc9-a695-d0d2c429c8cd service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:31.467755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008279f-780b-4cc9-a695-d0d2c429c8cd service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:31.468748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008279f-780b-4cc9-a695-d0d2c429c8cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:31.470745 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3008279f-780b-4cc9-a695-d0d2c429c8cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:31.575702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:31.620892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008279f-780b-4cc9-a695-d0d2c429c8cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:31.621446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008279f-780b-4cc9-a695-d0d2c429c8cd service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:31.644274 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 669/1331] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:33.945386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cbb68-c98e-4a18-8169-eb0d06671614 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:33.946129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cbb68-c98e-4a18-8169-eb0d06671614 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:33.946129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cbb68-c98e-4a18-8169-eb0d06671614 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:33.947314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 663/1332] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:33.954940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eada7b3-81c2-4e0d-ab52-e0c3497aab5c None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:33.955514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eada7b3-81c2-4e0d-ab52-e0c3497aab5c None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:33.955514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eada7b3-81c2-4e0d-ab52-e0c3497aab5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:34.029352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-4eada7b3-81c2-4e0d-ab52-e0c3497aab5c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:34.031187 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4eada7b3-81c2-4e0d-ab52-e0c3497aab5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:34.140116 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 670/1333] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:34.426871 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cbb68-c98e-4a18-8169-eb0d06671614 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:34.428452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-eb9cbb68-c98e-4a18-8169-eb0d06671614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:34.506055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:34.542482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cbb68-c98e-4a18-8169-eb0d06671614 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:34.621115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2724269-b2ae-4acc-bdb1-34ed89b572fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:34.621724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2724269-b2ae-4acc-bdb1-34ed89b572fd service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:34.622002 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2724269-b2ae-4acc-bdb1-34ed89b572fd service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:34.622271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2724269-b2ae-4acc-bdb1-34ed89b572fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:34.624512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b2724269-b2ae-4acc-bdb1-34ed89b572fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:34.744975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2724269-b2ae-4acc-bdb1-34ed89b572fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:34.746471 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2724269-b2ae-4acc-bdb1-34ed89b572fd service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:34.758078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:34.768409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 664/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:36.832045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d2c7-b703-4cf1-bda0-b3df728121b8 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:36.832650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d2c7-b703-4cf1-bda0-b3df728121b8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:36.832650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d2c7-b703-4cf1-bda0-b3df728121b8 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:36.834006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 671/1335] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:36.840065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60d1c2-8f40-407b-a113-7ab6363c596e None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:36.840909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60d1c2-8f40-407b-a113-7ab6363c596e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:36.840909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60d1c2-8f40-407b-a113-7ab6363c596e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:36.917388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ca60d1c2-8f40-407b-a113-7ab6363c596e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:36.918617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ca60d1c2-8f40-407b-a113-7ab6363c596e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:37.026400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 665/1336] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:40:37.320177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98d2c7-b703-4cf1-bda0-b3df728121b8 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:37.321722 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7c98d2c7-b703-4cf1-bda0-b3df728121b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:37.365233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:37.433464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98d2c7-b703-4cf1-bda0-b3df728121b8 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:37.509362 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51517a3-46fe-4e92-9449-89333c0301ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:37.509966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51517a3-46fe-4e92-9449-89333c0301ea service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:37.510243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51517a3-46fe-4e92-9449-89333c0301ea service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:37.510513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51517a3-46fe-4e92-9449-89333c0301ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:37.512147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d51517a3-46fe-4e92-9449-89333c0301ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:37.598338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:37.624966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51517a3-46fe-4e92-9449-89333c0301ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:37.626414 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51517a3-46fe-4e92-9449-89333c0301ea service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:37.645430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 672/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:39.994694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc520445-a83f-426b-badf-2f3834842d2b None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:39.995406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc520445-a83f-426b-badf-2f3834842d2b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:39.995406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc520445-a83f-426b-badf-2f3834842d2b None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:39.996593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 666/1338] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:40.002286 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa5c6c-da1a-4658-9708-d30819471a97 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:40.002744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa5c6c-da1a-4658-9708-d30819471a97 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:40.002857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa5c6c-da1a-4658-9708-d30819471a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:40.073263 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-beaa5c6c-da1a-4658-9708-d30819471a97 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:40.074509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-beaa5c6c-da1a-4658-9708-d30819471a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:40.176027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 673/1339] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:40.475819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc520445-a83f-426b-badf-2f3834842d2b None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:40.477394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-dc520445-a83f-426b-badf-2f3834842d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:40.558067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:40.586904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc520445-a83f-426b-badf-2f3834842d2b None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:40.665719 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea818969-14e2-4f69-9119-75fa34ab5332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:40.666329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea818969-14e2-4f69-9119-75fa34ab5332 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:40.666603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea818969-14e2-4f69-9119-75fa34ab5332 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:40.666868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea818969-14e2-4f69-9119-75fa34ab5332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:40.668537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ea818969-14e2-4f69-9119-75fa34ab5332 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:40.790608 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea818969-14e2-4f69-9119-75fa34ab5332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:40.792111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea818969-14e2-4f69-9119-75fa34ab5332 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:40.816703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 667/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:45.156193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf1f66-78f4-4d6a-a8ee-8100253ea5c5 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:45.156825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf1f66-78f4-4d6a-a8ee-8100253ea5c5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:45.156825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf1f66-78f4-4d6a-a8ee-8100253ea5c5 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:45.158124 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 674/1341] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:45.165032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e287b-b499-4c5f-a08b-eb9188ffcc7b None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:45.165523 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e287b-b499-4c5f-a08b-eb9188ffcc7b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:45.165612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e287b-b499-4c5f-a08b-eb9188ffcc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:45.171099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:45.243844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f60e287b-b499-4c5f-a08b-eb9188ffcc7b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:45.245537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f60e287b-b499-4c5f-a08b-eb9188ffcc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:45.361214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 668/1342] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:45.662218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf1f66-78f4-4d6a-a8ee-8100253ea5c5 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:45.663883 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0fdf1f66-78f4-4d6a-a8ee-8100253ea5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:45.717313 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:45.781193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf1f66-78f4-4d6a-a8ee-8100253ea5c5 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:45.865921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d19e4b-9cac-4dfd-b1b8-18c02aa08570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:45.866541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d19e4b-9cac-4dfd-b1b8-18c02aa08570 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:45.866818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d19e4b-9cac-4dfd-b1b8-18c02aa08570 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:45.867093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d19e4b-9cac-4dfd-b1b8-18c02aa08570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:45.868765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-29d19e4b-9cac-4dfd-b1b8-18c02aa08570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:45.972219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:45.994036 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d19e4b-9cac-4dfd-b1b8-18c02aa08570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:45.995794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d19e4b-9cac-4dfd-b1b8-18c02aa08570 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:46.021764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 675/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:48.784261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a2570-e547-4f8d-810b-4c2d5fab1107 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:48.784857 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a2570-e547-4f8d-810b-4c2d5fab1107 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:48.784857 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a2570-e547-4f8d-810b-4c2d5fab1107 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:48.786188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 669/1344] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:48.792351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4b5ce-d306-4733-b3a0-010ca23ad69e None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:48.792928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4b5ce-d306-4733-b3a0-010ca23ad69e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:48.792928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4b5ce-d306-4733-b3a0-010ca23ad69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:48.872231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-faa4b5ce-d306-4733-b3a0-010ca23ad69e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:48.875614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-faa4b5ce-d306-4733-b3a0-010ca23ad69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:49.003303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 676/1345] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:49.290724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a2570-e547-4f8d-810b-4c2d5fab1107 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:49.292630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-666a2570-e547-4f8d-810b-4c2d5fab1107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:49.399142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:49.419508 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a2570-e547-4f8d-810b-4c2d5fab1107 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:49.507319 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ad710-0b3d-4f75-be55-a857751f6dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:49.507956 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ad710-0b3d-4f75-be55-a857751f6dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:49.508237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ad710-0b3d-4f75-be55-a857751f6dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:49.508813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ad710-0b3d-4f75-be55-a857751f6dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:49.511292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-786ad710-0b3d-4f75-be55-a857751f6dd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:49.645796 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ad710-0b3d-4f75-be55-a857751f6dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:49.647227 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ad710-0b3d-4f75-be55-a857751f6dd9 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:49.669361 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 670/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:54.400254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7ae4e-791f-48eb-be6f-9468c6b589c7 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:54.400947 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7ae4e-791f-48eb-be6f-9468c6b589c7 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:54.400947 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7ae4e-791f-48eb-be6f-9468c6b589c7 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:54.402176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 677/1347] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:40:54.408343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaf5cc-7f52-427c-9177-2de1076e93eb None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:54.408857 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaf5cc-7f52-427c-9177-2de1076e93eb None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:54.408857 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaf5cc-7f52-427c-9177-2de1076e93eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:54.425773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:54.502255 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4afaf5cc-7f52-427c-9177-2de1076e93eb None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:54.503528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4afaf5cc-7f52-427c-9177-2de1076e93eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:54.640168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 671/1348] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:54.954602 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7ae4e-791f-48eb-be6f-9468c6b589c7 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:54.956028 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-98d7ae4e-791f-48eb-be6f-9468c6b589c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:55.023072 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:55.075011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7ae4e-791f-48eb-be6f-9468c6b589c7 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:55.153518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc9555-b685-4f04-b623-01cde535faf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:55.154671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc9555-b685-4f04-b623-01cde535faf0 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:55.154943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc9555-b685-4f04-b623-01cde535faf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:55.155421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc9555-b685-4f04-b623-01cde535faf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:55.157081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-efbc9555-b685-4f04-b623-01cde535faf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:55.292523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:55.298396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc9555-b685-4f04-b623-01cde535faf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:55.300075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc9555-b685-4f04-b623-01cde535faf0 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:55.320769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 678/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:00.606703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f28a68-8e60-4110-be44-016e96b7663d None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:00.607882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f28a68-8e60-4110-be44-016e96b7663d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:00.607882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f28a68-8e60-4110-be44-016e96b7663d None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:00.609470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 672/1350] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 09:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:00.615565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c9aaa-f7eb-41da-877a-bac210bc643f None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:00.616224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c9aaa-f7eb-41da-877a-bac210bc643f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:00.616224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c9aaa-f7eb-41da-877a-bac210bc643f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:00.767542 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-1c6c9aaa-f7eb-41da-877a-bac210bc643f None None] MFA Rules not processed for user `560df3eb05f14b6489158ab053f282ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:00.769900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1c6c9aaa-f7eb-41da-877a-bac210bc643f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:00.943895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 679/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 09:41:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:04.635529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=59 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:05.792570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69063028-e4c1-4689-8843-56ae30c822bf None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:05.793227 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69063028-e4c1-4689-8843-56ae30c822bf None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:05.793227 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69063028-e4c1-4689-8843-56ae30c822bf None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:05.794728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 673/1352] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 09:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:41:05.799895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32bc81-cca0-45fd-a4b8-78846f8cc3d2 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:05.800547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32bc81-cca0-45fd-a4b8-78846f8cc3d2 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:05.800547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32bc81-cca0-45fd-a4b8-78846f8cc3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:05.865506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:05.900024 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-da32bc81-cca0-45fd-a4b8-78846f8cc3d2 None None] MFA Rules not processed for user `560df3eb05f14b6489158ab053f282ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:05.902363 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-da32bc81-cca0-45fd-a4b8-78846f8cc3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:06.081727 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 680/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 09:41:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:07.694728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5713fe1-111d-4b49-b7dc-b5280ee4eebf None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:07.695301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5713fe1-111d-4b49-b7dc-b5280ee4eebf None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:07.695301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5713fe1-111d-4b49-b7dc-b5280ee4eebf None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:07.696683 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 674/1354] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 09:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:07.704376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e010596-3bb7-4d3a-a2ee-241763a2bacd None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:07.704907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e010596-3bb7-4d3a-a2ee-241763a2bacd None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:07.705025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e010596-3bb7-4d3a-a2ee-241763a2bacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:07.821859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-8e010596-3bb7-4d3a-a2ee-241763a2bacd None None] MFA Rules not processed for user `560df3eb05f14b6489158ab053f282ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:07.823357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-8e010596-3bb7-4d3a-a2ee-241763a2bacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:07.873685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:07.985536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 681/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 09:41:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:11.533442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f7a9a-8c01-4533-9629-b2369c35a06a None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:11.534025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f7a9a-8c01-4533-9629-b2369c35a06a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:11.534025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f7a9a-8c01-4533-9629-b2369c35a06a None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:11.535570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 675/1356] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:11.542369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92a9c7-9a5d-47d3-bf43-baaf87cc837f None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:11.543712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92a9c7-9a5d-47d3-bf43-baaf87cc837f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:11.546444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92a9c7-9a5d-47d3-bf43-baaf87cc837f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:11.633235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0b92a9c7-9a5d-47d3-bf43-baaf87cc837f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:11.634360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0b92a9c7-9a5d-47d3-bf43-baaf87cc837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:11.759104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 682/1357] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:41:12.104817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d8c55-e77d-460b-a74a-912a70f7a555 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:12.107292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d8c55-e77d-460b-a74a-912a70f7a555 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:12.108030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d8c55-e77d-460b-a74a-912a70f7a555 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:12.111256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 676/1358] 10.222.0.36 () {54 vars in 962 bytes} [Wed Aug 31 09:41:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:12.120959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6749f-9b32-46bf-b3da-0308897f822c None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:12.122622 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6749f-9b32-46bf-b3da-0308897f822c None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:12.124092 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6749f-9b32-46bf-b3da-0308897f822c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:12.148619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:12.254838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b2e6749f-9b32-46bf-b3da-0308897f822c None None] MFA Rules not processed for user `ef72bb95992c4a21ae2dbb2a08c51421`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:12.256184 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b2e6749f-9b32-46bf-b3da-0308897f822c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:12.431927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 683/1359] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Aug 31 09:41:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:12.446647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d8c55-e77d-460b-a74a-912a70f7a555 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:12.448817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-659d8c55-e77d-460b-a74a-912a70f7a555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:12.667799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d8c55-e77d-460b-a74a-912a70f7a555 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:12.768696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:12.824684 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0c4e9-c623-4259-9a61-9ba573f18af2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:12.826438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0c4e9-c623-4259-9a61-9ba573f18af2 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:12.826742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0c4e9-c623-4259-9a61-9ba573f18af2 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:12.827002 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0c4e9-c623-4259-9a61-9ba573f18af2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:12.829996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6de0c4e9-c623-4259-9a61-9ba573f18af2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:12.956969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0c4e9-c623-4259-9a61-9ba573f18af2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:12.958834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0c4e9-c623-4259-9a61-9ba573f18af2 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:12.986957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 677/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:15.323646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d812252-a0fb-43ef-8f48-365cbe20c9f9 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:15.324312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d812252-a0fb-43ef-8f48-365cbe20c9f9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:15.324312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d812252-a0fb-43ef-8f48-365cbe20c9f9 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:15.325952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 684/1361] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:15.331192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672fa29-c751-49ea-8b03-4e5ffbe82576 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:15.332967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672fa29-c751-49ea-8b03-4e5ffbe82576 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:15.332967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672fa29-c751-49ea-8b03-4e5ffbe82576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:15.404732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-2672fa29-c751-49ea-8b03-4e5ffbe82576 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:15.406052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2672fa29-c751-49ea-8b03-4e5ffbe82576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:15.446772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:15.527845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 678/1362] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:15.805890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d812252-a0fb-43ef-8f48-365cbe20c9f9 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:15.807674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1d812252-a0fb-43ef-8f48-365cbe20c9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:15.893627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:16.021314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d812252-a0fb-43ef-8f48-365cbe20c9f9 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:16.198481 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97363b6-6706-4ec9-951b-cdae9bc4d608 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:16.199721 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97363b6-6706-4ec9-951b-cdae9bc4d608 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:16.200816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97363b6-6706-4ec9-951b-cdae9bc4d608 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:16.201312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97363b6-6706-4ec9-951b-cdae9bc4d608 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:16.203224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a97363b6-6706-4ec9-951b-cdae9bc4d608 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:16.235271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:16.339586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97363b6-6706-4ec9-951b-cdae9bc4d608 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:16.341498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97363b6-6706-4ec9-951b-cdae9bc4d608 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:16.382010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 685/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:18.708844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b3bd4-f356-4c33-94c4-5b1e59532ee9 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:18.709866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b3bd4-f356-4c33-94c4-5b1e59532ee9 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:18.710375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b3bd4-f356-4c33-94c4-5b1e59532ee9 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:18.711824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 679/1364] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:18.718610 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c30e5-c249-4af2-bdfa-effd4b17245b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:18.719462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c30e5-c249-4af2-bdfa-effd4b17245b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:18.719914 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c30e5-c249-4af2-bdfa-effd4b17245b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:18.792699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ac2c30e5-c249-4af2-bdfa-effd4b17245b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:18.794619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ac2c30e5-c249-4af2-bdfa-effd4b17245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:18.903355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:18.915516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 686/1365] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:41:19.202338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2641a2-8923-4cf4-8b3c-73a37aeaa767 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:19.202643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2641a2-8923-4cf4-8b3c-73a37aeaa767 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:19.202911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2641a2-8923-4cf4-8b3c-73a37aeaa767 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:19.204414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 680/1366] 10.222.0.36 () {54 vars in 962 bytes} [Wed Aug 31 09:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:19.210447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4af21-b35b-4f6c-828d-f4efeae7b7cf None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:19.210889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4af21-b35b-4f6c-828d-f4efeae7b7cf None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:19.211279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4af21-b35b-4f6c-828d-f4efeae7b7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:19.296268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-7af4af21-b35b-4f6c-828d-f4efeae7b7cf None None] MFA Rules not processed for user `ef72bb95992c4a21ae2dbb2a08c51421`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:19.297734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7af4af21-b35b-4f6c-828d-f4efeae7b7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:19.441966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 687/1367] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Aug 31 09:41:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:19.450656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2641a2-8923-4cf4-8b3c-73a37aeaa767 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:19.452294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3a2641a2-8923-4cf4-8b3c-73a37aeaa767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:19.619376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2641a2-8923-4cf4-8b3c-73a37aeaa767 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:19.648640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:19.775366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208c7b3-abae-412a-b62c-803367d89fdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:19.775366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208c7b3-abae-412a-b62c-803367d89fdc service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:19.776145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208c7b3-abae-412a-b62c-803367d89fdc service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:19.776145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208c7b3-abae-412a-b62c-803367d89fdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:19.777890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d208c7b3-abae-412a-b62c-803367d89fdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:19.899799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208c7b3-abae-412a-b62c-803367d89fdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:19.901461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208c7b3-abae-412a-b62c-803367d89fdc service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:19.924724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 681/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:20.456269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40a5f1-4c58-4ca2-ad0d-5b944ebd0759 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:20.457082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40a5f1-4c58-4ca2-ad0d-5b944ebd0759 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:20.457082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40a5f1-4c58-4ca2-ad0d-5b944ebd0759 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:20.458463 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 688/1369] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:20.463871 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084177e4-99c5-48a3-a9ea-ef50013ad110 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:20.464188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084177e4-99c5-48a3-a9ea-ef50013ad110 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:20.464433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084177e4-99c5-48a3-a9ea-ef50013ad110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:20.510573 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:20.581132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-084177e4-99c5-48a3-a9ea-ef50013ad110 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:20.583246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-084177e4-99c5-48a3-a9ea-ef50013ad110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:20.738127 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 682/1370] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:21.334085 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb9213-c5b5-446f-80dc-28e95bdc9bf5 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:21.334085 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb9213-c5b5-446f-80dc-28e95bdc9bf5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:21.334085 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb9213-c5b5-446f-80dc-28e95bdc9bf5 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:21.337090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 689/1371] 10.222.0.36 () {54 vars in 964 bytes} [Wed Aug 31 09:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:21.344487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50c316-fc84-4be3-b33d-64c9f7c68b74 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:21.344980 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50c316-fc84-4be3-b33d-64c9f7c68b74 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:21.345063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50c316-fc84-4be3-b33d-64c9f7c68b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:21.441855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-6a50c316-fc84-4be3-b33d-64c9f7c68b74 None None] MFA Rules not processed for user `21d8ade66b7e4a1880c961c5e13adf04`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:21.443266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6a50c316-fc84-4be3-b33d-64c9f7c68b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:21.498200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:21.533942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 683/1372] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Aug 31 09:41:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:21.542001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb9213-c5b5-446f-80dc-28e95bdc9bf5 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:21.543686 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-97cb9213-c5b5-446f-80dc-28e95bdc9bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:21.601764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:21.665604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb9213-c5b5-446f-80dc-28e95bdc9bf5 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:21.760617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b37c6-5ee3-469c-8fbe-5e586f62f011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:21.761331 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b37c6-5ee3-469c-8fbe-5e586f62f011 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:21.761483 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b37c6-5ee3-469c-8fbe-5e586f62f011 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:21.761782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b37c6-5ee3-469c-8fbe-5e586f62f011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:21.764341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-fb3b37c6-5ee3-469c-8fbe-5e586f62f011 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:21.882014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:21.899910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b37c6-5ee3-469c-8fbe-5e586f62f011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:21.901451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b37c6-5ee3-469c-8fbe-5e586f62f011 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:21.923885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 690/1373] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:22.013262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236fc58-6ecf-49b6-8dc4-812955b0f8ef None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:22.013529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236fc58-6ecf-49b6-8dc4-812955b0f8ef None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:22.014521 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236fc58-6ecf-49b6-8dc4-812955b0f8ef None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:22.015155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 684/1374] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:41:22.020515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6686d7-e8b1-4d4e-a44d-518ffdf11fda None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:22.020805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6686d7-e8b1-4d4e-a44d-518ffdf11fda None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:22.021084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6686d7-e8b1-4d4e-a44d-518ffdf11fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:22.100945 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-4b6686d7-e8b1-4d4e-a44d-518ffdf11fda None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:22.102323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4b6686d7-e8b1-4d4e-a44d-518ffdf11fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:22.203104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 691/1375] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:41:22.521866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236fc58-6ecf-49b6-8dc4-812955b0f8ef None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:22.523543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3236fc58-6ecf-49b6-8dc4-812955b0f8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:22.707001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236fc58-6ecf-49b6-8dc4-812955b0f8ef None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:22.772261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:22.844100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728645d-a630-4dda-8816-279c6d7d0780 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:22.844708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728645d-a630-4dda-8816-279c6d7d0780 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:22.844985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728645d-a630-4dda-8816-279c6d7d0780 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:22.845253 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728645d-a630-4dda-8816-279c6d7d0780 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:22.846877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b728645d-a630-4dda-8816-279c6d7d0780 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:22.960094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728645d-a630-4dda-8816-279c6d7d0780 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:22.961826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728645d-a630-4dda-8816-279c6d7d0780 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:22.984750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 685/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:24.975798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0264b6-25f2-4ce4-a9bc-88fdde401a07 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:24.976315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0264b6-25f2-4ce4-a9bc-88fdde401a07 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:24.976315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0264b6-25f2-4ce4-a9bc-88fdde401a07 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:24.977682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 692/1377] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:24.983645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1dd0a-ca09-4a05-a252-2bd557851b43 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:24.984187 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1dd0a-ca09-4a05-a252-2bd557851b43 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:24.984187 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1dd0a-ca09-4a05-a252-2bd557851b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:25.062837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-1cc1dd0a-ca09-4a05-a252-2bd557851b43 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:25.062837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1cc1dd0a-ca09-4a05-a252-2bd557851b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:25.071114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:25.196082 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 686/1378] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:25.495410 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0264b6-25f2-4ce4-a9bc-88fdde401a07 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:25.497568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0b0264b6-25f2-4ce4-a9bc-88fdde401a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:25.578616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:25.720269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0264b6-25f2-4ce4-a9bc-88fdde401a07 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:25.863946 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab18de-af14-43bd-b30a-80b1ec938937 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:25.864574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab18de-af14-43bd-b30a-80b1ec938937 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:25.864852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab18de-af14-43bd-b30a-80b1ec938937 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:25.865122 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab18de-af14-43bd-b30a-80b1ec938937 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:25.866890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e2ab18de-af14-43bd-b30a-80b1ec938937 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:25.894884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:25.994813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab18de-af14-43bd-b30a-80b1ec938937 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:25.996413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab18de-af14-43bd-b30a-80b1ec938937 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:26.019950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 693/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:28.081953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381cf78-3d0c-46d5-8539-05ee80a185ff None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:28.082379 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381cf78-3d0c-46d5-8539-05ee80a185ff None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:28.082485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381cf78-3d0c-46d5-8539-05ee80a185ff None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:28.083873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 687/1380] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:28.090241 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7575a-e9d3-4b0d-95dd-641eb0eb88e3 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:28.090830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7575a-e9d3-4b0d-95dd-641eb0eb88e3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:28.090830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7575a-e9d3-4b0d-95dd-641eb0eb88e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:28.175885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-65a7575a-e9d3-4b0d-95dd-641eb0eb88e3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:28.177243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-65a7575a-e9d3-4b0d-95dd-641eb0eb88e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:28.278533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:28.286422 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 694/1381] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:28.588681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381cf78-3d0c-46d5-8539-05ee80a185ff None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:28.591002 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3381cf78-3d0c-46d5-8539-05ee80a185ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:28.776431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:28.798379 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381cf78-3d0c-46d5-8539-05ee80a185ff None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:28.945984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effd608-262b-4789-8591-9a221805bb34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:28.945984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effd608-262b-4789-8591-9a221805bb34 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:28.945984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effd608-262b-4789-8591-9a221805bb34 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:28.945984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effd608-262b-4789-8591-9a221805bb34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:28.946506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0effd608-262b-4789-8591-9a221805bb34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:29.066920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effd608-262b-4789-8591-9a221805bb34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:29.069704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effd608-262b-4789-8591-9a221805bb34 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:29.084855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:29.096268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 688/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:31.115509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b31ad1-eac6-40d4-a6ce-2e5055025e9d None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:31.116187 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b31ad1-eac6-40d4-a6ce-2e5055025e9d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:31.116187 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b31ad1-eac6-40d4-a6ce-2e5055025e9d None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:31.117396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 695/1383] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:41:31.123708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a88ef-d6df-44c4-bcf6-d1a6be95abda None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:31.124201 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a88ef-d6df-44c4-bcf6-d1a6be95abda None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:31.124298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a88ef-d6df-44c4-bcf6-d1a6be95abda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:31.202239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a80a88ef-d6df-44c4-bcf6-d1a6be95abda None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:31.203486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a80a88ef-d6df-44c4-bcf6-d1a6be95abda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:31.310323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 689/1384] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:31.604839 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b31ad1-eac6-40d4-a6ce-2e5055025e9d None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:31.606423 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-35b31ad1-eac6-40d4-a6ce-2e5055025e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:31.796447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b31ad1-eac6-40d4-a6ce-2e5055025e9d None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:31.896032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:31.963785 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0089d-c6cb-4bf4-9cf9-f2027216e713 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:31.964608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0089d-c6cb-4bf4-9cf9-f2027216e713 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:31.965017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0089d-c6cb-4bf4-9cf9-f2027216e713 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:31.965950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0089d-c6cb-4bf4-9cf9-f2027216e713 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:31.968480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-70b0089d-c6cb-4bf4-9cf9-f2027216e713 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:32.110889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0089d-c6cb-4bf4-9cf9-f2027216e713 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:32.113252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0089d-c6cb-4bf4-9cf9-f2027216e713 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:32.140537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 696/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:34.585217 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a71e76-905f-41ab-bb68-c9fc5852cc55 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:34.585785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a71e76-905f-41ab-bb68-c9fc5852cc55 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:34.586215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a71e76-905f-41ab-bb68-c9fc5852cc55 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:34.588202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 690/1386] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:34.595367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a697a9-3815-4a61-aeee-dcfb97dbc0fc None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:34.595871 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a697a9-3815-4a61-aeee-dcfb97dbc0fc None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:34.596006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a697a9-3815-4a61-aeee-dcfb97dbc0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:34.697139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-74a697a9-3815-4a61-aeee-dcfb97dbc0fc None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:34.698390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-74a697a9-3815-4a61-aeee-dcfb97dbc0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:34.741163 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:34.829177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 697/1387] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:35.125880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a71e76-905f-41ab-bb68-c9fc5852cc55 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:35.128190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-70a71e76-905f-41ab-bb68-c9fc5852cc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:35.216924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:35.304727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a71e76-905f-41ab-bb68-c9fc5852cc55 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:35.447002 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073de94-e3b1-4f31-9852-a3eb1e508257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:35.447983 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073de94-e3b1-4f31-9852-a3eb1e508257 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:35.448494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073de94-e3b1-4f31-9852-a3eb1e508257 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:35.448944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073de94-e3b1-4f31-9852-a3eb1e508257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:35.452352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e073de94-e3b1-4f31-9852-a3eb1e508257 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:35.501050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:35.562723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073de94-e3b1-4f31-9852-a3eb1e508257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:35.564404 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073de94-e3b1-4f31-9852-a3eb1e508257 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:35.583003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 691/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:37.377037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d018e7c-2b89-47b0-aaa1-d11cacb5674d None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:37.377554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d018e7c-2b89-47b0-aaa1-d11cacb5674d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:37.377651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d018e7c-2b89-47b0-aaa1-d11cacb5674d None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:37.378972 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 698/1389] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:37.384861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a1196-4131-4d5c-be21-74ad7e773bb3 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:37.385328 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a1196-4131-4d5c-be21-74ad7e773bb3 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:37.385410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a1196-4131-4d5c-be21-74ad7e773bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:37.458434 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-9a5a1196-4131-4d5c-be21-74ad7e773bb3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:37.459700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9a5a1196-4131-4d5c-be21-74ad7e773bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:37.566725 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 692/1390] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:37.616767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15f982-9bc3-4672-bc01-1e5c82af6e06 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:37.617036 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15f982-9bc3-4672-bc01-1e5c82af6e06 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:37.617428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15f982-9bc3-4672-bc01-1e5c82af6e06 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:37.619182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 699/1391] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:37.625268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baecc4-0d58-44e5-8512-e64a76086228 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:37.625554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baecc4-0d58-44e5-8512-e64a76086228 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:37.625993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baecc4-0d58-44e5-8512-e64a76086228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:37.642570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:37.698234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-62baecc4-0d58-44e5-8512-e64a76086228 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:37.699545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-62baecc4-0d58-44e5-8512-e64a76086228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:37.804003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 693/1392] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:37.873220 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be254ad7-26d2-407c-946b-b1adab09ae87 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:37.873438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be254ad7-26d2-407c-946b-b1adab09ae87 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:37.873904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be254ad7-26d2-407c-946b-b1adab09ae87 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:37.875092 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 700/1393] 10.222.0.36 () {54 vars in 967 bytes} [Wed Aug 31 09:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:37.880428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548914d-69d8-4223-a1e5-f68bbc007567 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:37.880863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548914d-69d8-4223-a1e5-f68bbc007567 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:37.880987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548914d-69d8-4223-a1e5-f68bbc007567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:37.961343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b548914d-69d8-4223-a1e5-f68bbc007567 None None] MFA Rules not processed for user `06fad9466df14827983dac2b4b844f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:37.962555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b548914d-69d8-4223-a1e5-f68bbc007567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:37.999506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:38.042131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 694/1394] 10.222.0.36 () {58 vars in 1064 bytes} [Wed Aug 31 09:41:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:38.050410 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be254ad7-26d2-407c-946b-b1adab09ae87 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:38.051995 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-be254ad7-26d2-407c-946b-b1adab09ae87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:38.093722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548914d-69d8-4223-a1e5-f68bbc007567 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:38.094912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b548914d-69d8-4223-a1e5-f68bbc007567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:38.164834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be254ad7-26d2-407c-946b-b1adab09ae87 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:38.221323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548914d-69d8-4223-a1e5-f68bbc007567 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:38.231564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:38.250617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe4537-6baf-43cd-b81c-683b9683bf00 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06fad9466df14827983dac2b4b844f3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:38.251612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe4537-6baf-43cd-b81c-683b9683bf00 service glance] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:38.251891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe4537-6baf-43cd-b81c-683b9683bf00 service glance] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:38.252635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe4537-6baf-43cd-b81c-683b9683bf00 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:38.254315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2dbe4537-6baf-43cd-b81c-683b9683bf00 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:38.291056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:38.346515 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc1a4e-b45a-4b21-9332-71d50ee3a612 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:38.347143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc1a4e-b45a-4b21-9332-71d50ee3a612 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:38.347430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc1a4e-b45a-4b21-9332-71d50ee3a612 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:38.347697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc1a4e-b45a-4b21-9332-71d50ee3a612 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:38.349353 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-64dc1a4e-b45a-4b21-9332-71d50ee3a612 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:38.372948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe4537-6baf-43cd-b81c-683b9683bf00 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:38.374392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe4537-6baf-43cd-b81c-683b9683bf00 service glance] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:38.394179 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 701/1395] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:38.443308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc1a4e-b45a-4b21-9332-71d50ee3a612 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:38.444906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc1a4e-b45a-4b21-9332-71d50ee3a612 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:38.463947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 695/1396] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:41:39.172508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ac655-4e08-4158-a51c-22e119ac3eb4 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:39.172803 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ac655-4e08-4158-a51c-22e119ac3eb4 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:39.173083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ac655-4e08-4158-a51c-22e119ac3eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:39.244424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-149ac655-4e08-4158-a51c-22e119ac3eb4 None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:39.245667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-149ac655-4e08-4158-a51c-22e119ac3eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:39.292287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:39.343672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 702/1397] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:41:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:39.393885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db955c3c-95e1-48e4-983a-e53d9c322ee9 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:39.394170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db955c3c-95e1-48e4-983a-e53d9c322ee9 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:39.394435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db955c3c-95e1-48e4-983a-e53d9c322ee9 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:39.395841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 696/1398] 10.222.0.36 () {54 vars in 943 bytes} [Wed Aug 31 09:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:39.403904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd8e16-7e3d-4fc0-9625-d408ad8cfeb8 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:39.404405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd8e16-7e3d-4fc0-9625-d408ad8cfeb8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:39.404588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd8e16-7e3d-4fc0-9625-d408ad8cfeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:39.491116 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-41bd8e16-7e3d-4fc0-9625-d408ad8cfeb8 None None] MFA Rules not processed for user `c5cf86ae159749cea0cc9783bffbfe75`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:39.492433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-41bd8e16-7e3d-4fc0-9625-d408ad8cfeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:39.616801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 703/1399] 10.222.0.36 () {58 vars in 1040 bytes} [Wed Aug 31 09:41:39 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:39.627716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db955c3c-95e1-48e4-983a-e53d9c322ee9 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:39.629257 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-db955c3c-95e1-48e4-983a-e53d9c322ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:39.707491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:39.772855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db955c3c-95e1-48e4-983a-e53d9c322ee9 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:39.879589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff582c6a-f4ed-47e3-9374-4043c82eaa5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:39.880069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff582c6a-f4ed-47e3-9374-4043c82eaa5c service swift] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:39.880340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff582c6a-f4ed-47e3-9374-4043c82eaa5c service swift] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:39.880606 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff582c6a-f4ed-47e3-9374-4043c82eaa5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:39.882242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ff582c6a-f4ed-47e3-9374-4043c82eaa5c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:39.946108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:39.971031 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff582c6a-f4ed-47e3-9374-4043c82eaa5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:39.972878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff582c6a-f4ed-47e3-9374-4043c82eaa5c service swift] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:39.982025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 697/1400] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:41:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 09:41:40.479422 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d685bc-2c09-4979-b608-1f643fabbe9d None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:40.479956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d685bc-2c09-4979-b608-1f643fabbe9d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:40.480038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d685bc-2c09-4979-b608-1f643fabbe9d None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:40.482340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 704/1401] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:40.488419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7773b9c-e343-43f4-9fc5-8b3977bf8a65 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:40.488871 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7773b9c-e343-43f4-9fc5-8b3977bf8a65 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:40.489161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7773b9c-e343-43f4-9fc5-8b3977bf8a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:40.567234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-e7773b9c-e343-43f4-9fc5-8b3977bf8a65 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:40.568367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e7773b9c-e343-43f4-9fc5-8b3977bf8a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:40.688717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 698/1402] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:41:40.990915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d685bc-2c09-4979-b608-1f643fabbe9d None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:40.992586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-54d685bc-2c09-4979-b608-1f643fabbe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:41.009681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:41.208859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d685bc-2c09-4979-b608-1f643fabbe9d None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:41.319016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:41.349991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcb22d-faa2-4302-83ac-90d9ff8f115f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:41.350599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcb22d-faa2-4302-83ac-90d9ff8f115f service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:41.350883 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcb22d-faa2-4302-83ac-90d9ff8f115f service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:41.351422 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcb22d-faa2-4302-83ac-90d9ff8f115f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:41.353054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-69bcb22d-faa2-4302-83ac-90d9ff8f115f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:41.456530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcb22d-faa2-4302-83ac-90d9ff8f115f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:41.458139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcb22d-faa2-4302-83ac-90d9ff8f115f service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:41.483743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 705/1403] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:43.514118 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef00d-29a5-40a4-bcc0-0b020bf00256 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:43.514716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef00d-29a5-40a4-bcc0-0b020bf00256 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:43.514716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef00d-29a5-40a4-bcc0-0b020bf00256 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:43.516097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 699/1404] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:43.522165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd1682-832a-4f98-850a-a96dc1a8af32 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:43.522625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd1682-832a-4f98-850a-a96dc1a8af32 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:43.522720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd1682-832a-4f98-850a-a96dc1a8af32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:43.602354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-50fd1682-832a-4f98-850a-a96dc1a8af32 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:43.603634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-50fd1682-832a-4f98-850a-a96dc1a8af32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:43.658808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:43.714498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 706/1405] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:41:44.005391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edef00d-29a5-40a4-bcc0-0b020bf00256 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:44.008248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6edef00d-29a5-40a4-bcc0-0b020bf00256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:44.073476 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:44.177656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edef00d-29a5-40a4-bcc0-0b020bf00256 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:44.319024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e5aab-29cc-4aee-a8cd-ae78e911e9cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:44.319537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e5aab-29cc-4aee-a8cd-ae78e911e9cc service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:44.319926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e5aab-29cc-4aee-a8cd-ae78e911e9cc service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:44.320130 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e5aab-29cc-4aee-a8cd-ae78e911e9cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:44.321672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7c5e5aab-29cc-4aee-a8cd-ae78e911e9cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:44.339548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:44.430554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e5aab-29cc-4aee-a8cd-ae78e911e9cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:44.432297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e5aab-29cc-4aee-a8cd-ae78e911e9cc service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:44.453869 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 700/1406] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:46.525034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5856f0-36a2-4db0-8bc2-8b6c0df7e0e8 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:46.525605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5856f0-36a2-4db0-8bc2-8b6c0df7e0e8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:46.525707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5856f0-36a2-4db0-8bc2-8b6c0df7e0e8 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:46.531152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 707/1407] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:46.534112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd20127-6a62-4af4-9c98-ea0c1993af70 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:46.534112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd20127-6a62-4af4-9c98-ea0c1993af70 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:46.534112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd20127-6a62-4af4-9c98-ea0c1993af70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:46.620170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ebd20127-6a62-4af4-9c98-ea0c1993af70 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:46.622042 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ebd20127-6a62-4af4-9c98-ea0c1993af70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:46.723670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:46.737206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 701/1408] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:47.037131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5856f0-36a2-4db0-8bc2-8b6c0df7e0e8 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:47.038669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7a5856f0-36a2-4db0-8bc2-8b6c0df7e0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:47.222058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5856f0-36a2-4db0-8bc2-8b6c0df7e0e8 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:47.264453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:47.362582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca730a7-3352-49ba-9b47-e3d206a2ff49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:47.363360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca730a7-3352-49ba-9b47-e3d206a2ff49 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:47.363644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca730a7-3352-49ba-9b47-e3d206a2ff49 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:47.364010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca730a7-3352-49ba-9b47-e3d206a2ff49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:47.368395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0ca730a7-3352-49ba-9b47-e3d206a2ff49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:47.489474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca730a7-3352-49ba-9b47-e3d206a2ff49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:47.492074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca730a7-3352-49ba-9b47-e3d206a2ff49 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:47.519088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 708/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:56.734980 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:57.516087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=83 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:02.506411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0c0d-6166-4a96-a3a0-9e229751e348 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:02.507062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0c0d-6166-4a96-a3a0-9e229751e348 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:02.507062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0c0d-6166-4a96-a3a0-9e229751e348 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:02.508574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 702/1410] 10.222.0.36 () {54 vars in 918 bytes} [Wed Aug 31 09:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:02.514679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a379f80-c451-4991-bb63-979bf5f4baa2 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:02.515235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a379f80-c451-4991-bb63-979bf5f4baa2 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:02.515354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a379f80-c451-4991-bb63-979bf5f4baa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:02.603788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-7a379f80-c451-4991-bb63-979bf5f4baa2 None None] MFA Rules not processed for user `560df3eb05f14b6489158ab053f282ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:02.605038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7a379f80-c451-4991-bb63-979bf5f4baa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:02.717153 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 709/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Aug 31 09:42:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:02.738893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cae5c-0ebe-42ac-a25b-f90431ecde2e None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:02.739219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cae5c-0ebe-42ac-a25b-f90431ecde2e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:02.739466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cae5c-0ebe-42ac-a25b-f90431ecde2e None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:02.740771 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 703/1412] 10.222.0.36 () {54 vars in 918 bytes} [Wed Aug 31 09:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:02.745513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d2e53-53a5-494e-b90a-12b3f11f5f90 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:02.745806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d2e53-53a5-494e-b90a-12b3f11f5f90 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:02.746867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d2e53-53a5-494e-b90a-12b3f11f5f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:02.820655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:02.823735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-4e4d2e53-53a5-494e-b90a-12b3f11f5f90 None None] MFA Rules not processed for user `ef72bb95992c4a21ae2dbb2a08c51421`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:02.826065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4e4d2e53-53a5-494e-b90a-12b3f11f5f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:02.941676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 710/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Aug 31 09:42:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:05.412725 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce750043-ad6f-4d34-b843-4dcad6c55811 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:05.413310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce750043-ad6f-4d34-b843-4dcad6c55811 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:05.413310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce750043-ad6f-4d34-b843-4dcad6c55811 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:05.414579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:05.420826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8152c-802e-41be-b0e6-d81dc8c3071b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:05.421329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8152c-802e-41be-b0e6-d81dc8c3071b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:05.421439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8152c-802e-41be-b0e6-d81dc8c3071b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:05.491715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-19b8152c-802e-41be-b0e6-d81dc8c3071b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:05.492950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-19b8152c-802e-41be-b0e6-d81dc8c3071b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:05.550731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:05.595876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:42:05.892249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce750043-ad6f-4d34-b843-4dcad6c55811 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:05.893840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ce750043-ad6f-4d34-b843-4dcad6c55811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:06.059565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce750043-ad6f-4d34-b843-4dcad6c55811 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:06.144798 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:06.183157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edd00c-716b-4efe-a89b-e379f4f76923 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:06.184305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edd00c-716b-4efe-a89b-e379f4f76923 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:06.184577 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edd00c-716b-4efe-a89b-e379f4f76923 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:06.184839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edd00c-716b-4efe-a89b-e379f4f76923 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:06.186454 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-46edd00c-716b-4efe-a89b-e379f4f76923 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:06.279956 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edd00c-716b-4efe-a89b-e379f4f76923 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:06.282166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edd00c-716b-4efe-a89b-e379f4f76923 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:06.302027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 705/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:07.232637 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbfa3c-33a1-4347-9b68-01df5d9fd859 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:07.233508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbfa3c-33a1-4347-9b68-01df5d9fd859 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:07.233508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbfa3c-33a1-4347-9b68-01df5d9fd859 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:07.235169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:42:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:07.240247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf177e-65d3-4cfb-a365-716868b8e75f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:07.240684 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf177e-65d3-4cfb-a365-716868b8e75f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:07.240791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf177e-65d3-4cfb-a365-716868b8e75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:07.319985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-e7bf177e-65d3-4cfb-a365-716868b8e75f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:07.321210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e7bf177e-65d3-4cfb-a365-716868b8e75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:07.349477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:07.427254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:07.577330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbfa3c-33a1-4347-9b68-01df5d9fd859 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:07.578859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7afbfa3c-33a1-4347-9b68-01df5d9fd859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:07.732111 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbfa3c-33a1-4347-9b68-01df5d9fd859 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:07.773575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:07.848257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda10f92-cfc8-4a92-a1f0-4db1799e5bc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:07.848870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda10f92-cfc8-4a92-a1f0-4db1799e5bc5 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:07.849147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda10f92-cfc8-4a92-a1f0-4db1799e5bc5 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:07.849418 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda10f92-cfc8-4a92-a1f0-4db1799e5bc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:07.851159 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-bda10f92-cfc8-4a92-a1f0-4db1799e5bc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:07.941864 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda10f92-cfc8-4a92-a1f0-4db1799e5bc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:07.943506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda10f92-cfc8-4a92-a1f0-4db1799e5bc5 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:07.962828 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 713/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:08.868708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab1764-76d8-42e5-a717-1b0b8143f49f None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:08.869263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab1764-76d8-42e5-a717-1b0b8143f49f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:08.869263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab1764-76d8-42e5-a717-1b0b8143f49f None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:08.871100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:42:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:08.876482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88772bc-f2ea-421c-af58-8243ca02bbbe None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:08.876948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88772bc-f2ea-421c-af58-8243ca02bbbe None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:08.877064 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88772bc-f2ea-421c-af58-8243ca02bbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:08.937452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:08.960697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e88772bc-f2ea-421c-af58-8243ca02bbbe None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:08.961938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e88772bc-f2ea-421c-af58-8243ca02bbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:09.074412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:09.228033 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab1764-76d8-42e5-a717-1b0b8143f49f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:09.229593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-46ab1764-76d8-42e5-a717-1b0b8143f49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:09.384580 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab1764-76d8-42e5-a717-1b0b8143f49f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:09.398073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:09.501518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc572a87-3616-49e6-9121-64ea06886fff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:09.502037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc572a87-3616-49e6-9121-64ea06886fff service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:09.502313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc572a87-3616-49e6-9121-64ea06886fff service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:09.502582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc572a87-3616-49e6-9121-64ea06886fff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:09.504251 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-cc572a87-3616-49e6-9121-64ea06886fff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:09.596802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc572a87-3616-49e6-9121-64ea06886fff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:09.598405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc572a87-3616-49e6-9121-64ea06886fff service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:09.617338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 708/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:10.497992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746e06a-e38d-4352-93e9-b1f1ad3c01d8 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:10.498557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746e06a-e38d-4352-93e9-b1f1ad3c01d8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:10.498557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746e06a-e38d-4352-93e9-b1f1ad3c01d8 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:10.499990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:10.505233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac221d8-4d1e-4fdc-9404-19484578986c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:10.505672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac221d8-4d1e-4fdc-9404-19484578986c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:10.505780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac221d8-4d1e-4fdc-9404-19484578986c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:10.526864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:10.585383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-0ac221d8-4d1e-4fdc-9404-19484578986c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:10.586609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0ac221d8-4d1e-4fdc-9404-19484578986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:10.686791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:10.842389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746e06a-e38d-4352-93e9-b1f1ad3c01d8 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:10.843884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2746e06a-e38d-4352-93e9-b1f1ad3c01d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:10.959031 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:10.998035 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746e06a-e38d-4352-93e9-b1f1ad3c01d8 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:11.115986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7735c4c-cd6a-4107-b6e5-8e219d8b00df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:11.116503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7735c4c-cd6a-4107-b6e5-8e219d8b00df service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:11.116780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7735c4c-cd6a-4107-b6e5-8e219d8b00df service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:11.117048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7735c4c-cd6a-4107-b6e5-8e219d8b00df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:11.118652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c7735c4c-cd6a-4107-b6e5-8e219d8b00df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:11.200406 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:11.215851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7735c4c-cd6a-4107-b6e5-8e219d8b00df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:11.217451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7735c4c-cd6a-4107-b6e5-8e219d8b00df service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:11.236461 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 716/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:12.112142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ed02f-28d0-454d-a36f-2a2865799cf6 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:12.112697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ed02f-28d0-454d-a36f-2a2865799cf6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:12.112697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ed02f-28d0-454d-a36f-2a2865799cf6 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:12.114021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:12.119002 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0128a-1ee6-486c-aac1-d10813b0241f None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:12.119521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0128a-1ee6-486c-aac1-d10813b0241f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:12.119621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0128a-1ee6-486c-aac1-d10813b0241f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:12.200159 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-2ef0128a-1ee6-486c-aac1-d10813b0241f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:12.201399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2ef0128a-1ee6-486c-aac1-d10813b0241f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:12.301129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:12.454243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ed02f-28d0-454d-a36f-2a2865799cf6 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:12.455711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7d5ed02f-28d0-454d-a36f-2a2865799cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:12.541264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:12.603838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ed02f-28d0-454d-a36f-2a2865799cf6 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:12.716878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266d7d0-a3db-454d-bf89-940f9bf0bfea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:12.717389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266d7d0-a3db-454d-bf89-940f9bf0bfea service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:12.717664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266d7d0-a3db-454d-bf89-940f9bf0bfea service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:12.717929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266d7d0-a3db-454d-bf89-940f9bf0bfea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:12.719551 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b266d7d0-a3db-454d-bf89-940f9bf0bfea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:12.769057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:12.812403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266d7d0-a3db-454d-bf89-940f9bf0bfea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:12.814056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266d7d0-a3db-454d-bf89-940f9bf0bfea service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:12.832118 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 711/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:13.652567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa0215-e740-47f2-b3e2-b66130d447ff None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:13.653164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa0215-e740-47f2-b3e2-b66130d447ff None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:13.653164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa0215-e740-47f2-b3e2-b66130d447ff None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:13.654494 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:13.659832 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4dc62-7e90-4ab0-b797-3f9f45e38e93 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:13.660285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4dc62-7e90-4ab0-b797-3f9f45e38e93 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:13.660404 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4dc62-7e90-4ab0-b797-3f9f45e38e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:13.738228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-3bd4dc62-7e90-4ab0-b797-3f9f45e38e93 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:13.739478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3bd4dc62-7e90-4ab0-b797-3f9f45e38e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:13.840880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:13.990892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa0215-e740-47f2-b3e2-b66130d447ff None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:13.992435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-63fa0215-e740-47f2-b3e2-b66130d447ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:14.044413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:14.143302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa0215-e740-47f2-b3e2-b66130d447ff None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:14.255899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab78869-49cc-467c-9742-2fb5fcb9bcc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:14.256422 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab78869-49cc-467c-9742-2fb5fcb9bcc7 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:14.256704 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab78869-49cc-467c-9742-2fb5fcb9bcc7 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:14.256971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab78869-49cc-467c-9742-2fb5fcb9bcc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:14.258849 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9ab78869-49cc-467c-9742-2fb5fcb9bcc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:14.266048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:14.361821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab78869-49cc-467c-9742-2fb5fcb9bcc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:14.363465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab78869-49cc-467c-9742-2fb5fcb9bcc7 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:14.383142 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 719/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:15.210481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28680f01-4d7c-42c7-a20e-1a8cd0954894 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:15.211058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28680f01-4d7c-42c7-a20e-1a8cd0954894 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:15.211058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28680f01-4d7c-42c7-a20e-1a8cd0954894 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:15.212396 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:15.219327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ba7b3-bdce-47d9-8617-b33479455500 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:15.219865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ba7b3-bdce-47d9-8617-b33479455500 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:15.219865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ba7b3-bdce-47d9-8617-b33479455500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:15.306862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-d14ba7b3-bdce-47d9-8617-b33479455500 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:15.308179 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d14ba7b3-bdce-47d9-8617-b33479455500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:15.383035 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:15.420091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:15.581298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28680f01-4d7c-42c7-a20e-1a8cd0954894 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:15.582816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-28680f01-4d7c-42c7-a20e-1a8cd0954894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:15.600833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:15.734701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28680f01-4d7c-42c7-a20e-1a8cd0954894 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:15.823427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:15.857722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f1757-8ce1-4648-a6ea-ecc66f29678c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:15.858331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f1757-8ce1-4648-a6ea-ecc66f29678c service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:15.858613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f1757-8ce1-4648-a6ea-ecc66f29678c service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:15.858877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f1757-8ce1-4648-a6ea-ecc66f29678c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:15.860512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d71f1757-8ce1-4648-a6ea-ecc66f29678c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:15.950721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f1757-8ce1-4648-a6ea-ecc66f29678c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:15.952391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f1757-8ce1-4648-a6ea-ecc66f29678c service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:15.973444 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 714/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:16.795316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec9608-c88a-4bb9-b19a-85674d3a1579 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:16.795872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec9608-c88a-4bb9-b19a-85674d3a1579 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:16.795872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec9608-c88a-4bb9-b19a-85674d3a1579 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:16.797466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:42:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:16.803155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29d7b0-75d1-49c0-8c45-6bb1760bf83b None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:16.803721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29d7b0-75d1-49c0-8c45-6bb1760bf83b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:16.803846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29d7b0-75d1-49c0-8c45-6bb1760bf83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:16.887443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-da29d7b0-75d1-49c0-8c45-6bb1760bf83b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:16.888669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-da29d7b0-75d1-49c0-8c45-6bb1760bf83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:16.942312 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:17.000776 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:17.154497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec9608-c88a-4bb9-b19a-85674d3a1579 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:17.156171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-95ec9608-c88a-4bb9-b19a-85674d3a1579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:17.311559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec9608-c88a-4bb9-b19a-85674d3a1579 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:17.367054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:17.429790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfadd0f-1e37-45a5-b42d-71d388e2e87a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:17.430399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfadd0f-1e37-45a5-b42d-71d388e2e87a service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:17.430674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfadd0f-1e37-45a5-b42d-71d388e2e87a service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:17.430941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfadd0f-1e37-45a5-b42d-71d388e2e87a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:17.432585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0bfadd0f-1e37-45a5-b42d-71d388e2e87a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:17.526194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfadd0f-1e37-45a5-b42d-71d388e2e87a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:17.528373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfadd0f-1e37-45a5-b42d-71d388e2e87a service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:17.547897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 722/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:18.373479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78fde9-1ba2-4285-80f6-a78dcc4f22e7 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:18.374041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78fde9-1ba2-4285-80f6-a78dcc4f22e7 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:18.374041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78fde9-1ba2-4285-80f6-a78dcc4f22e7 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:18.375322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:18.380493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d5a6b-afe0-42fb-8d83-576338e77600 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:18.380962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d5a6b-afe0-42fb-8d83-576338e77600 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:18.381060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d5a6b-afe0-42fb-8d83-576338e77600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:18.459858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-f65d5a6b-afe0-42fb-8d83-576338e77600 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:18.461096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-f65d5a6b-afe0-42fb-8d83-576338e77600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:18.463182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:18.562038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:18.713295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78fde9-1ba2-4285-80f6-a78dcc4f22e7 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:18.715634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ae78fde9-1ba2-4285-80f6-a78dcc4f22e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:18.862830 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78fde9-1ba2-4285-80f6-a78dcc4f22e7 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:18.890436 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:18.982246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada769f2-3d2b-46c2-9087-cac0411b7784 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:18.982764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada769f2-3d2b-46c2-9087-cac0411b7784 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:18.983039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada769f2-3d2b-46c2-9087-cac0411b7784 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:18.983351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada769f2-3d2b-46c2-9087-cac0411b7784 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:18.984972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ada769f2-3d2b-46c2-9087-cac0411b7784 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:19.076196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada769f2-3d2b-46c2-9087-cac0411b7784 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:19.077795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada769f2-3d2b-46c2-9087-cac0411b7784 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:19.096662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 717/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:20.958762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3b834-a213-47c0-96df-4909a0b1083e None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:20.959360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3b834-a213-47c0-96df-4909a0b1083e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:20.959360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3b834-a213-47c0-96df-4909a0b1083e None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:20.960693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:20.966070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ae800-a374-4353-820e-5b84dda5df88 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:20.966522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ae800-a374-4353-820e-5b84dda5df88 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:20.966627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ae800-a374-4353-820e-5b84dda5df88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:21.007842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:21.036296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-049ae800-a374-4353-820e-5b84dda5df88 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:21.037509 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-049ae800-a374-4353-820e-5b84dda5df88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:21.140887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:21.435728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3b834-a213-47c0-96df-4909a0b1083e None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:21.437291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d3a3b834-a213-47c0-96df-4909a0b1083e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:21.547888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3b834-a213-47c0-96df-4909a0b1083e None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:21.583210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:21.625367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512800b-ec38-4179-8d5a-60b299853462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:21.625985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512800b-ec38-4179-8d5a-60b299853462 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:21.626266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512800b-ec38-4179-8d5a-60b299853462 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:21.626541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512800b-ec38-4179-8d5a-60b299853462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:21.628198 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7512800b-ec38-4179-8d5a-60b299853462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:21.743631 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512800b-ec38-4179-8d5a-60b299853462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:21.745089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512800b-ec38-4179-8d5a-60b299853462 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:21.764703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 725/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:23.908741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74edb3-37ba-414a-9e00-45aec5055726 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:23.909338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74edb3-37ba-414a-9e00-45aec5055726 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:23.909338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74edb3-37ba-414a-9e00-45aec5055726 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:23.910627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:23.915985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990b02b-7d95-47c5-8591-8bab3a9a9b02 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:23.916441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990b02b-7d95-47c5-8591-8bab3a9a9b02 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:23.916553 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990b02b-7d95-47c5-8591-8bab3a9a9b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:23.985617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b990b02b-7d95-47c5-8591-8bab3a9a9b02 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:23.986880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b990b02b-7d95-47c5-8591-8bab3a9a9b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:23.999593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:24.084138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:24.372694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74edb3-37ba-414a-9e00-45aec5055726 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:24.374267 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1d74edb3-37ba-414a-9e00-45aec5055726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:24.485307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74edb3-37ba-414a-9e00-45aec5055726 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:24.497803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:24.562350 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0aba71-013f-4e23-9bcd-df0864c8cc1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:24.562954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aba71-013f-4e23-9bcd-df0864c8cc1b service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:24.563251 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aba71-013f-4e23-9bcd-df0864c8cc1b service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:24.563515 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aba71-013f-4e23-9bcd-df0864c8cc1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:24.565116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0e0aba71-013f-4e23-9bcd-df0864c8cc1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:24.675801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0aba71-013f-4e23-9bcd-df0864c8cc1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:24.677229 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0aba71-013f-4e23-9bcd-df0864c8cc1b service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:24.696799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 720/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:26.968710 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad7bcb-2dec-43ad-a3fe-21ecf50af055 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:26.969319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad7bcb-2dec-43ad-a3fe-21ecf50af055 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:26.969319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad7bcb-2dec-43ad-a3fe-21ecf50af055 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:26.970657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:26.977831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1407fa-0403-4a6f-ba70-1cddaa88ea65 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:26.978348 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1407fa-0403-4a6f-ba70-1cddaa88ea65 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:26.978447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1407fa-0403-4a6f-ba70-1cddaa88ea65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:27.030420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:27.049412 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ff1407fa-0403-4a6f-ba70-1cddaa88ea65 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:27.050629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ff1407fa-0403-4a6f-ba70-1cddaa88ea65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:27.150119 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:27.441898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad7bcb-2dec-43ad-a3fe-21ecf50af055 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:27.443475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0aad7bcb-2dec-43ad-a3fe-21ecf50af055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:27.552883 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad7bcb-2dec-43ad-a3fe-21ecf50af055 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:27.596085 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:27.631204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98a73f-de89-4ab2-9392-9149aada9ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:27.631810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98a73f-de89-4ab2-9392-9149aada9ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:27.632093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98a73f-de89-4ab2-9392-9149aada9ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:27.632369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98a73f-de89-4ab2-9392-9149aada9ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:27.634257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2a98a73f-de89-4ab2-9392-9149aada9ab5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:27.754930 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98a73f-de89-4ab2-9392-9149aada9ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:27.756391 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98a73f-de89-4ab2-9392-9149aada9ab5 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:27.775427 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 728/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:29.860554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4970a3-4321-41f4-b840-a731d471ace6 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:29.861185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4970a3-4321-41f4-b840-a731d471ace6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:29.861185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4970a3-4321-41f4-b840-a731d471ace6 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:29.862465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:29.868564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d28dba-3819-41db-9bd8-2c9606aa3946 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:29.869149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d28dba-3819-41db-9bd8-2c9606aa3946 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:29.869149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d28dba-3819-41db-9bd8-2c9606aa3946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:29.946681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-94d28dba-3819-41db-9bd8-2c9606aa3946 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:29.948218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-94d28dba-3819-41db-9bd8-2c9606aa3946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:29.970158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:30.050520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:30.341983 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4970a3-4321-41f4-b840-a731d471ace6 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:30.343615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-dd4970a3-4321-41f4-b840-a731d471ace6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:30.454585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4970a3-4321-41f4-b840-a731d471ace6 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:30.475700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:30.533710 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d321110-0de7-402b-989f-47c14de0a0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:30.534314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d321110-0de7-402b-989f-47c14de0a0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:30.534587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d321110-0de7-402b-989f-47c14de0a0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:30.534853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d321110-0de7-402b-989f-47c14de0a0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:30.536485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2d321110-0de7-402b-989f-47c14de0a0a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:30.647664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d321110-0de7-402b-989f-47c14de0a0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:30.649100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d321110-0de7-402b-989f-47c14de0a0a0 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:30.667288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 723/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:32.714297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3f876-d926-4100-87ac-4868a9475cdc None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:32.714852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3f876-d926-4100-87ac-4868a9475cdc None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:32.715054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3f876-d926-4100-87ac-4868a9475cdc None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:32.716537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:32.722047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a5560-f47c-4e6c-9655-f57ccc90effe None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:32.722538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a5560-f47c-4e6c-9655-f57ccc90effe None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:32.722637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a5560-f47c-4e6c-9655-f57ccc90effe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:32.794382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:32.797424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f61a5560-f47c-4e6c-9655-f57ccc90effe None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:32.799292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f61a5560-f47c-4e6c-9655-f57ccc90effe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:32.906705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:42:33.204912 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3f876-d926-4100-87ac-4868a9475cdc None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:33.206509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-87f3f876-d926-4100-87ac-4868a9475cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:33.314540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3f876-d926-4100-87ac-4868a9475cdc None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:33.364345 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:33.391164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39fc05-b07c-4d2c-a6bd-3d8c55503fa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:33.391789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39fc05-b07c-4d2c-a6bd-3d8c55503fa5 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:33.392066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39fc05-b07c-4d2c-a6bd-3d8c55503fa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:33.392363 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39fc05-b07c-4d2c-a6bd-3d8c55503fa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:33.393975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7e39fc05-b07c-4d2c-a6bd-3d8c55503fa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:33.511299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39fc05-b07c-4d2c-a6bd-3d8c55503fa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:33.512745 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39fc05-b07c-4d2c-a6bd-3d8c55503fa5 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:33.532527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 731/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:35.524442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8079e48-a9f9-4f01-8ee8-aa34d3aa4125 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:35.525045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8079e48-a9f9-4f01-8ee8-aa34d3aa4125 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:35.525045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8079e48-a9f9-4f01-8ee8-aa34d3aa4125 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:35.526318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:35.533467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8fa19-b066-49a8-b26a-0996c79214ff None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:35.534261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8fa19-b066-49a8-b26a-0996c79214ff None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:35.534261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8fa19-b066-49a8-b26a-0996c79214ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:35.603920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-07c8fa19-b066-49a8-b26a-0996c79214ff None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:35.605164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-07c8fa19-b066-49a8-b26a-0996c79214ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:35.636950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:35.707267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:35.995107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8079e48-a9f9-4f01-8ee8-aa34d3aa4125 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:35.996954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a8079e48-a9f9-4f01-8ee8-aa34d3aa4125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:36.107942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8079e48-a9f9-4f01-8ee8-aa34d3aa4125 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:36.138802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:36.183438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3f765-79bb-474a-bd29-9f77b74b2b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:36.185044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3f765-79bb-474a-bd29-9f77b74b2b18 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:36.185320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3f765-79bb-474a-bd29-9f77b74b2b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:36.185585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3f765-79bb-474a-bd29-9f77b74b2b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:36.187226 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-02c3f765-79bb-474a-bd29-9f77b74b2b18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:36.297757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3f765-79bb-474a-bd29-9f77b74b2b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:36.299100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3f765-79bb-474a-bd29-9f77b74b2b18 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:36.317549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 726/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:42:38.333434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595f2ff-5fc4-4d81-9d98-9c92ad07d495 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:38.333959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595f2ff-5fc4-4d81-9d98-9c92ad07d495 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:38.333959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595f2ff-5fc4-4d81-9d98-9c92ad07d495 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:38.335317 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:38.341557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650644e4-770c-49d2-84ff-3779cfcc4c3d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:38.342083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650644e4-770c-49d2-84ff-3779cfcc4c3d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:38.342083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650644e4-770c-49d2-84ff-3779cfcc4c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:38.410892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-650644e4-770c-49d2-84ff-3779cfcc4c3d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:38.412136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-650644e4-770c-49d2-84ff-3779cfcc4c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:38.420942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:38.511879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:38.807350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595f2ff-5fc4-4d81-9d98-9c92ad07d495 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:38.809518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1595f2ff-5fc4-4d81-9d98-9c92ad07d495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:38.927863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595f2ff-5fc4-4d81-9d98-9c92ad07d495 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:38.991335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:39.012622 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb238c4a-886f-4953-9b7e-40ef59c12692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:39.013246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb238c4a-886f-4953-9b7e-40ef59c12692 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:39.013527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb238c4a-886f-4953-9b7e-40ef59c12692 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:39.013799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb238c4a-886f-4953-9b7e-40ef59c12692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:39.016301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-cb238c4a-886f-4953-9b7e-40ef59c12692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:39.146250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb238c4a-886f-4953-9b7e-40ef59c12692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:39.148018 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb238c4a-886f-4953-9b7e-40ef59c12692 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:39.167359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 734/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:41.225278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8e900-d089-42a3-abe7-3fb0dff66c30 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:41.225819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8e900-d089-42a3-abe7-3fb0dff66c30 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:41.225819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8e900-d089-42a3-abe7-3fb0dff66c30 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:41.227200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:41.233368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c672f1-65f3-4462-baf1-d71dc87fb491 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:41.233829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c672f1-65f3-4462-baf1-d71dc87fb491 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:41.233939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c672f1-65f3-4462-baf1-d71dc87fb491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:41.300711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-c3c672f1-65f3-4462-baf1-d71dc87fb491 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:41.301921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c3c672f1-65f3-4462-baf1-d71dc87fb491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:41.343102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:41.404479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:41.706646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8e900-d089-42a3-abe7-3fb0dff66c30 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:41.708246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c2e8e900-d089-42a3-abe7-3fb0dff66c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:41.821898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8e900-d089-42a3-abe7-3fb0dff66c30 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:41.864720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:41.904925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd51ac-da8d-4428-b237-b75ea8e8f1b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:41.905546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd51ac-da8d-4428-b237-b75ea8e8f1b5 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:41.905816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd51ac-da8d-4428-b237-b75ea8e8f1b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:41.906089 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd51ac-da8d-4428-b237-b75ea8e8f1b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:41.907776 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-aabd51ac-da8d-4428-b237-b75ea8e8f1b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:42.029094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd51ac-da8d-4428-b237-b75ea8e8f1b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:42.030503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd51ac-da8d-4428-b237-b75ea8e8f1b5 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:42.048839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 729/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:42:44.005648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde118b7-4da6-449c-bf8e-b24c8e27dd06 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:44.006335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde118b7-4da6-449c-bf8e-b24c8e27dd06 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:44.006335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde118b7-4da6-449c-bf8e-b24c8e27dd06 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:44.007874 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:44.013867 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a9c66-64a0-4f97-9720-d1b589a3d78d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:44.014334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a9c66-64a0-4f97-9720-d1b589a3d78d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:44.014444 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a9c66-64a0-4f97-9720-d1b589a3d78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:44.082028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b00a9c66-64a0-4f97-9720-d1b589a3d78d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:44.083253 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b00a9c66-64a0-4f97-9720-d1b589a3d78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:44.100895 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:44.181372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:44.473141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde118b7-4da6-449c-bf8e-b24c8e27dd06 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:44.474698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-bde118b7-4da6-449c-bf8e-b24c8e27dd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:44.586898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde118b7-4da6-449c-bf8e-b24c8e27dd06 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:44.668511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:44.677894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aac813-7ecb-403b-bf8f-d5df721bba35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:44.678523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aac813-7ecb-403b-bf8f-d5df721bba35 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:44.678801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aac813-7ecb-403b-bf8f-d5df721bba35 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:44.679067 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aac813-7ecb-403b-bf8f-d5df721bba35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:44.680837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-57aac813-7ecb-403b-bf8f-d5df721bba35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:44.807272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aac813-7ecb-403b-bf8f-d5df721bba35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:44.809524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aac813-7ecb-403b-bf8f-d5df721bba35 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:44.828636 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 737/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:46.791876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2e6b9-9f0b-4619-b69d-62c1f66172dc None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:46.792647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2e6b9-9f0b-4619-b69d-62c1f66172dc None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:46.792647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2e6b9-9f0b-4619-b69d-62c1f66172dc None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:46.794025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:46.799821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1463c-3812-46fd-b297-e5ff297d7f31 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:46.800310 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1463c-3812-46fd-b297-e5ff297d7f31 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:46.800417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1463c-3812-46fd-b297-e5ff297d7f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:46.869323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-78a1463c-3812-46fd-b297-e5ff297d7f31 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:46.871083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-78a1463c-3812-46fd-b297-e5ff297d7f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:46.920780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:46.974595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:47.272885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2e6b9-9f0b-4619-b69d-62c1f66172dc None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:47.274549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-69a2e6b9-9f0b-4619-b69d-62c1f66172dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:47.385591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2e6b9-9f0b-4619-b69d-62c1f66172dc None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:47.440321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:47.469484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922a7e4-620e-4cd8-aed3-2097de097ba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:47.470078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a7e4-620e-4cd8-aed3-2097de097ba0 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:47.470357 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a7e4-620e-4cd8-aed3-2097de097ba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:47.470629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a7e4-620e-4cd8-aed3-2097de097ba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:47.472294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1922a7e4-620e-4cd8-aed3-2097de097ba0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:47.608607 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922a7e4-620e-4cd8-aed3-2097de097ba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:47.610045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922a7e4-620e-4cd8-aed3-2097de097ba0 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:47.628425 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 732/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:56.974011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=22 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:57.626037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=66 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:07.579068 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a059a-a58d-47ee-a452-5d766c1c38da None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:07.579624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a059a-a58d-47ee-a452-5d766c1c38da None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:07.579765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a059a-a58d-47ee-a452-5d766c1c38da None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:07.581070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 739/1471] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:43:07.586487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf49463-1269-43e4-81bb-d26c553bb56b None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:07.586915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf49463-1269-43e4-81bb-d26c553bb56b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:07.587030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf49463-1269-43e4-81bb-d26c553bb56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:07.658267 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-bdf49463-1269-43e4-81bb-d26c553bb56b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:43:07.659522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-bdf49463-1269-43e4-81bb-d26c553bb56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:07.759401 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 733/1472] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:43:08.061834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2e99e-b77f-4a55-a4ac-56d3fe27f570 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:08.062128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2e99e-b77f-4a55-a4ac-56d3fe27f570 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:08.062410 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2e99e-b77f-4a55-a4ac-56d3fe27f570 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:08.063796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 740/1473] 10.222.0.36 () {54 vars in 967 bytes} [Wed Aug 31 09:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:43:08.068622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2673d5-cfd9-43b0-af8a-47f98cafddc2 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:08.068907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2673d5-cfd9-43b0-af8a-47f98cafddc2 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:08.069310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2673d5-cfd9-43b0-af8a-47f98cafddc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:08.146469 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-fe2673d5-cfd9-43b0-af8a-47f98cafddc2 None None] MFA Rules not processed for user `06fad9466df14827983dac2b4b844f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:43:08.147718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fe2673d5-cfd9-43b0-af8a-47f98cafddc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:08.168590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:08.224858 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 734/1474] 10.222.0.36 () {58 vars in 1064 bytes} [Wed Aug 31 09:43:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:43:08.233873 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2e99e-b77f-4a55-a4ac-56d3fe27f570 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:43:08.235432 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9ea2e99e-b77f-4a55-a4ac-56d3fe27f570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:08.343204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2e99e-b77f-4a55-a4ac-56d3fe27f570 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:43:08.418644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3e2a3-d032-4c0a-abde-14926a518d0f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06fad9466df14827983dac2b4b844f3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:43:08.419294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3e2a3-d032-4c0a-abde-14926a518d0f service glance] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:08.419579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3e2a3-d032-4c0a-abde-14926a518d0f service glance] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:08.419854 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3e2a3-d032-4c0a-abde-14926a518d0f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:08.421473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-32c3e2a3-d032-4c0a-abde-14926a518d0f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:08.477228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:08.533899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3e2a3-d032-4c0a-abde-14926a518d0f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:43:08.535364 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3e2a3-d032-4c0a-abde-14926a518d0f service glance] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:43:08.674311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 741/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:43:10.746560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c85376-1c8c-4071-ab2a-8143cfd4b366 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:10.747105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c85376-1c8c-4071-ab2a-8143cfd4b366 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:10.747105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c85376-1c8c-4071-ab2a-8143cfd4b366 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:10.748370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 735/1476] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:43:10.753661 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25481a13-33ec-46da-9377-ac6fe3bdc1d9 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:10.754150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25481a13-33ec-46da-9377-ac6fe3bdc1d9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:10.754243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25481a13-33ec-46da-9377-ac6fe3bdc1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:10.825088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-25481a13-33ec-46da-9377-ac6fe3bdc1d9 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:43:10.826318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-25481a13-33ec-46da-9377-ac6fe3bdc1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:10.925720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 742/1477] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:43:11.205664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c85376-1c8c-4071-ab2a-8143cfd4b366 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:43:11.207364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c0c85376-1c8c-4071-ab2a-8143cfd4b366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:11.342889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c85376-1c8c-4071-ab2a-8143cfd4b366 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:43:11.392744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:11.441225 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e280d11-d03f-4221-a9fc-dea6d73d5471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:43:11.441832 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e280d11-d03f-4221-a9fc-dea6d73d5471 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:11.442108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e280d11-d03f-4221-a9fc-dea6d73d5471 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:11.442377 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e280d11-d03f-4221-a9fc-dea6d73d5471 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:11.443015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e280d11-d03f-4221-a9fc-dea6d73d5471 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:43:11.444826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: WARNING py.warnings [None req-3e280d11-d03f-4221-a9fc-dea6d73d5471 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:43:11.444826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: warnings.warn(msg) Aug 31 09:43:11.444826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: Aug 31 09:43:11.445970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e280d11-d03f-4221-a9fc-dea6d73d5471 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:43:11.471824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 736/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 09:43:11 2022] GET /identity/v3/projects => generated 2553 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:43:13.490040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde7a75-1dfb-4e65-a8d4-ea5379ca96e4 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:13.490664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde7a75-1dfb-4e65-a8d4-ea5379ca96e4 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:13.490664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde7a75-1dfb-4e65-a8d4-ea5379ca96e4 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:13.492083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 743/1479] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:43:13.498643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35a4a6-f751-48f7-9598-e55d37d7ceff None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:13.499203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35a4a6-f751-48f7-9598-e55d37d7ceff None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:13.499320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35a4a6-f751-48f7-9598-e55d37d7ceff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:13.569792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-fe35a4a6-f751-48f7-9598-e55d37d7ceff None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:43:13.571022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fe35a4a6-f751-48f7-9598-e55d37d7ceff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:13.670287 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 737/1480] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:43:13.969875 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde7a75-1dfb-4e65-a8d4-ea5379ca96e4 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:43:13.971493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4cde7a75-1dfb-4e65-a8d4-ea5379ca96e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:13.981195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:14.119136 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde7a75-1dfb-4e65-a8d4-ea5379ca96e4 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:43:14.201321 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:14.237729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f62aa3-636d-4a09-9be7-fef103019bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:43:14.238242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f62aa3-636d-4a09-9be7-fef103019bcd service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:14.238518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f62aa3-636d-4a09-9be7-fef103019bcd service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:14.238784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f62aa3-636d-4a09-9be7-fef103019bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:14.240426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e8f62aa3-636d-4a09-9be7-fef103019bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:14.336209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f62aa3-636d-4a09-9be7-fef103019bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:43:14.337813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f62aa3-636d-4a09-9be7-fef103019bcd service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:43:14.359352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 744/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:43:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:43:16.497245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10dcd0-d400-4bcb-8804-69976d34fad7 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:16.497865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10dcd0-d400-4bcb-8804-69976d34fad7 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:16.497865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10dcd0-d400-4bcb-8804-69976d34fad7 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:16.499070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 738/1482] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:43:16.504614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3852678-cf82-4191-a3bb-fbe45d8c3b38 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:16.505098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3852678-cf82-4191-a3bb-fbe45d8c3b38 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:16.505279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3852678-cf82-4191-a3bb-fbe45d8c3b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:16.578597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-d3852678-cf82-4191-a3bb-fbe45d8c3b38 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:43:16.579923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d3852678-cf82-4191-a3bb-fbe45d8c3b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:16.627989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:16.681697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 745/1483] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:43:16.976031 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10dcd0-d400-4bcb-8804-69976d34fad7 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:43:16.977555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5d10dcd0-d400-4bcb-8804-69976d34fad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:17.004341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:17.091701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10dcd0-d400-4bcb-8804-69976d34fad7 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:43:17.170673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbff99-2b93-4398-8357-d4e6d77aa9e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:43:17.171378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbff99-2b93-4398-8357-d4e6d77aa9e2 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:17.171660 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbff99-2b93-4398-8357-d4e6d77aa9e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:17.171958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbff99-2b93-4398-8357-d4e6d77aa9e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:17.173583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e6cbff99-2b93-4398-8357-d4e6d77aa9e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:17.257225 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:17.293979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbff99-2b93-4398-8357-d4e6d77aa9e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:43:17.295451 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbff99-2b93-4398-8357-d4e6d77aa9e2 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:43:17.314305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 739/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:43:19.334054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75d42c-b7e8-4c46-b603-a4955a86267a None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:19.334612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75d42c-b7e8-4c46-b603-a4955a86267a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:19.334612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75d42c-b7e8-4c46-b603-a4955a86267a None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:19.335952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 746/1485] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:43:19.341364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52848932-1c86-4ca9-b3a5-4e3b1bbe89d3 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:19.341802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52848932-1c86-4ca9-b3a5-4e3b1bbe89d3 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:19.341923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52848932-1c86-4ca9-b3a5-4e3b1bbe89d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:19.411072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-52848932-1c86-4ca9-b3a5-4e3b1bbe89d3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:43:19.412372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-52848932-1c86-4ca9-b3a5-4e3b1bbe89d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:19.509945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 740/1486] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:43:19.812756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75d42c-b7e8-4c46-b603-a4955a86267a None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:43:19.814291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-dc75d42c-b7e8-4c46-b603-a4955a86267a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:19.919602 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75d42c-b7e8-4c46-b603-a4955a86267a None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:43:19.992443 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:19.994524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11dc449-520f-4249-8a9b-de05b7a9676d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:43:19.995203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11dc449-520f-4249-8a9b-de05b7a9676d service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:19.995688 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11dc449-520f-4249-8a9b-de05b7a9676d service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:19.996142 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11dc449-520f-4249-8a9b-de05b7a9676d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:19.998082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d11dc449-520f-4249-8a9b-de05b7a9676d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:20.118256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11dc449-520f-4249-8a9b-de05b7a9676d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:43:20.120284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11dc449-520f-4249-8a9b-de05b7a9676d service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:43:20.139239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 747/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:43:22.251954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4c4b0-da23-495a-9a4c-8e22eed84192 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:22.252643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4c4b0-da23-495a-9a4c-8e22eed84192 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:22.252643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4c4b0-da23-495a-9a4c-8e22eed84192 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:22.253817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 741/1488] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:43:22.259877 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b72a96-9a7e-4bf9-b9a5-e64b6168a657 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:22.260876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b72a96-9a7e-4bf9-b9a5-e64b6168a657 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:22.261110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b72a96-9a7e-4bf9-b9a5-e64b6168a657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:22.328418 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-03b72a96-9a7e-4bf9-b9a5-e64b6168a657 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:43:22.329631 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-03b72a96-9a7e-4bf9-b9a5-e64b6168a657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:22.384722 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:22.432932 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 748/1489] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:43:22.733513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4c4b0-da23-495a-9a4c-8e22eed84192 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:43:22.735264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-cfa4c4b0-da23-495a-9a4c-8e22eed84192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:22.772845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:22.843574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4c4b0-da23-495a-9a4c-8e22eed84192 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:43:22.920117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc0e86-df31-4de9-9124-c63ed1301e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:43:22.921004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc0e86-df31-4de9-9124-c63ed1301e29 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:22.921004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc0e86-df31-4de9-9124-c63ed1301e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:22.921249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc0e86-df31-4de9-9124-c63ed1301e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:22.922868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2bdc0e86-df31-4de9-9124-c63ed1301e29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:23.008314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:23.039810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc0e86-df31-4de9-9124-c63ed1301e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:43:23.041253 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc0e86-df31-4de9-9124-c63ed1301e29 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:43:23.061587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 742/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:43:25.291891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aad75-7204-425c-b522-05744ee3fca2 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:25.292527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aad75-7204-425c-b522-05744ee3fca2 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:25.292639 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aad75-7204-425c-b522-05744ee3fca2 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:25.294027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 749/1491] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 09:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:43:25.299796 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523574f-53c6-484f-b2b9-c0044880580a None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:25.300261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523574f-53c6-484f-b2b9-c0044880580a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:25.300371 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523574f-53c6-484f-b2b9-c0044880580a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:25.368150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a523574f-53c6-484f-b2b9-c0044880580a None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:43:25.369284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a523574f-53c6-484f-b2b9-c0044880580a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:25.472352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 743/1492] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 09:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:43:25.764139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862aad75-7204-425c-b522-05744ee3fca2 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:43:25.765679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-862aad75-7204-425c-b522-05744ee3fca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:25.868585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862aad75-7204-425c-b522-05744ee3fca2 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:43:25.944051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8309d7-2858-410b-8e6a-85f9e9817fb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:43:25.944645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8309d7-2858-410b-8e6a-85f9e9817fb4 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:43:25.944917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8309d7-2858-410b-8e6a-85f9e9817fb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:43:25.945182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8309d7-2858-410b-8e6a-85f9e9817fb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:43:25.946752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7f8309d7-2858-410b-8e6a-85f9e9817fb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:43:25.959683 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:26.062230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8309d7-2858-410b-8e6a-85f9e9817fb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:43:26.063611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8309d7-2858-410b-8e6a-85f9e9817fb4 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:43:26.083647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 750/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:43:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:43:35.470532 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=82 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:36.080323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=50 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:40.061243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbcf6d-9637-4134-95b7-f2db41d43ab4 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:40.061838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbcf6d-9637-4134-95b7-f2db41d43ab4 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:40.061838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbcf6d-9637-4134-95b7-f2db41d43ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:40.138364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f8bbcf6d-9637-4134-95b7-f2db41d43ab4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:40.139589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f8bbcf6d-9637-4134-95b7-f2db41d43ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:40.235631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 744/1494] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:40.243689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe4482-4cfd-46fe-a14f-b16f6b5f8c17 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:40.244188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe4482-4cfd-46fe-a14f-b16f6b5f8c17 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:40.244295 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe4482-4cfd-46fe-a14f-b16f6b5f8c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:40.322650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-fbfe4482-4cfd-46fe-a14f-b16f6b5f8c17 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:40.323809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-fbfe4482-4cfd-46fe-a14f-b16f6b5f8c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:40.420902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 751/1495] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:40.425609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbcf6d-9637-4134-95b7-f2db41d43ab4 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:40.426808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f8bbcf6d-9637-4134-95b7-f2db41d43ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:40.438619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbcf6d-9637-4134-95b7-f2db41d43ab4 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:40.504651 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:40.538986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ba2e2-c502-40d1-8f77-8f83f98f3451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:40.540112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ba2e2-c502-40d1-8f77-8f83f98f3451 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:40.540544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ba2e2-c502-40d1-8f77-8f83f98f3451 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:40.540922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ba2e2-c502-40d1-8f77-8f83f98f3451 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:40.541565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ba2e2-c502-40d1-8f77-8f83f98f3451 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:44:40.541866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ba2e2-c502-40d1-8f77-8f83f98f3451 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:40.542881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: WARNING py.warnings [None req-356ba2e2-c502-40d1-8f77-8f83f98f3451 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:40.542881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: warnings.warn(msg) Aug 31 09:44:40.542881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: Aug 31 09:44:40.543800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ba2e2-c502-40d1-8f77-8f83f98f3451 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:40.556134 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 745/1496] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:44:40.564474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe4482-4cfd-46fe-a14f-b16f6b5f8c17 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:40.573612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe4482-4cfd-46fe-a14f-b16f6b5f8c17 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:40.633760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:40.666997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552e02f-1988-4a1c-9ed2-e8eed263267b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:40.667613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552e02f-1988-4a1c-9ed2-e8eed263267b admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:40.667886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552e02f-1988-4a1c-9ed2-e8eed263267b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:40.668152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552e02f-1988-4a1c-9ed2-e8eed263267b admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:40.668772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e552e02f-1988-4a1c-9ed2-e8eed263267b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:40.670523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: WARNING py.warnings [None req-e552e02f-1988-4a1c-9ed2-e8eed263267b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:40.670523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: warnings.warn(msg) Aug 31 09:44:40.670523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: Aug 31 09:44:40.671736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e552e02f-1988-4a1c-9ed2-e8eed263267b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:40.728516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 752/1497] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:44:40 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:44:40.732763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ba2e2-c502-40d1-8f77-8f83f98f3451 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:40.734312 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-356ba2e2-c502-40d1-8f77-8f83f98f3451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:40.769074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ba2e2-c502-40d1-8f77-8f83f98f3451 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:40.869044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7bc88-3f20-4db8-b31b-bfc76d11de43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:40.869553 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7bc88-3f20-4db8-b31b-bfc76d11de43 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:40.869826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7bc88-3f20-4db8-b31b-bfc76d11de43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:40.870090 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7bc88-3f20-4db8-b31b-bfc76d11de43 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:40.870733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7bc88-3f20-4db8-b31b-bfc76d11de43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:40.871756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: WARNING py.warnings [None req-f1e7bc88-3f20-4db8-b31b-bfc76d11de43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:40.871756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: warnings.warn(msg) Aug 31 09:44:40.871756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: Aug 31 09:44:40.872947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7bc88-3f20-4db8-b31b-bfc76d11de43 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:40.916332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 746/1498] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:44:40 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:44:40.920675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552e02f-1988-4a1c-9ed2-e8eed263267b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:40.922169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e552e02f-1988-4a1c-9ed2-e8eed263267b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:40.984462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552e02f-1988-4a1c-9ed2-e8eed263267b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:41.067994 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:41.079897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332a8aa-2acd-4d65-8a9a-25bb4adb45f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:41.080513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332a8aa-2acd-4d65-8a9a-25bb4adb45f9 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:41.080827 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332a8aa-2acd-4d65-8a9a-25bb4adb45f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:41.081100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332a8aa-2acd-4d65-8a9a-25bb4adb45f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:41.081750 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332a8aa-2acd-4d65-8a9a-25bb4adb45f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:41.082755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: WARNING py.warnings [None req-2332a8aa-2acd-4d65-8a9a-25bb4adb45f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:41.082755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: warnings.warn(msg) Aug 31 09:44:41.082755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: Aug 31 09:44:41.083689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332a8aa-2acd-4d65-8a9a-25bb4adb45f9 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:41.096112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 753/1499] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:44:41.100157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7bc88-3f20-4db8-b31b-bfc76d11de43 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:41.101674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f1e7bc88-3f20-4db8-b31b-bfc76d11de43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:41.113297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7bc88-3f20-4db8-b31b-bfc76d11de43 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:41.143810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:41.216541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cca72b-0bda-4129-990f-d3a1e4d72be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:41.217140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cca72b-0bda-4129-990f-d3a1e4d72be6 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:41.217413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cca72b-0bda-4129-990f-d3a1e4d72be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:41.217674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cca72b-0bda-4129-990f-d3a1e4d72be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:41.218300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cca72b-0bda-4129-990f-d3a1e4d72be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:41.219332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: WARNING py.warnings [None req-34cca72b-0bda-4129-990f-d3a1e4d72be6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:41.219332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: warnings.warn(msg) Aug 31 09:44:41.219332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: Aug 31 09:44:41.220212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cca72b-0bda-4129-990f-d3a1e4d72be6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:41.233395 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 747/1500] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:44:41.237826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332a8aa-2acd-4d65-8a9a-25bb4adb45f9 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:41.239393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2332a8aa-2acd-4d65-8a9a-25bb4adb45f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:41.273970 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332a8aa-2acd-4d65-8a9a-25bb4adb45f9 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:41.370562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6d1a7-8bce-4529-8c97-77c443f53730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:41.371256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6d1a7-8bce-4529-8c97-77c443f53730 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:41.371537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6d1a7-8bce-4529-8c97-77c443f53730 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:41.371808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6d1a7-8bce-4529-8c97-77c443f53730 admin admin] PATH_INFO: `/v3/projects/165ec418946f4a9bb4d4d12d9cd25f67/users/25ac0ff73f7543cfb86754dc3fb1c1cf/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:41.393711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6d1a7-8bce-4529-8c97-77c443f53730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=165ec418946f4a9bb4d4d12d9cd25f67, user_id=25ac0ff73f7543cfb86754dc3fb1c1cf, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:41.394819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: WARNING py.warnings [None req-d2d6d1a7-8bce-4529-8c97-77c443f53730 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:41.394819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: warnings.warn(msg) Aug 31 09:44:41.394819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: Aug 31 09:44:41.396788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6d1a7-8bce-4529-8c97-77c443f53730 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:41.421379 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 754/1501] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:44:41 2022] PUT /identity/v3/projects/165ec418946f4a9bb4d4d12d9cd25f67/users/25ac0ff73f7543cfb86754dc3fb1c1cf/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:44:41.434670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1b8bd-edaf-4bc4-85f7-3f15aafc11e6 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:41.434964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1b8bd-edaf-4bc4-85f7-3f15aafc11e6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:41.435289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1b8bd-edaf-4bc4-85f7-3f15aafc11e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:41.493773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-88e1b8bd-edaf-4bc4-85f7-3f15aafc11e6 None None] MFA Rules not processed for user `25ac0ff73f7543cfb86754dc3fb1c1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:41.494996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-88e1b8bd-edaf-4bc4-85f7-3f15aafc11e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:41.577116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 748/1502] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:44:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:41.590812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febefc1-8ee6-49fa-8112-48a1b95433b3 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:41.591102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febefc1-8ee6-49fa-8112-48a1b95433b3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:41.591409 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febefc1-8ee6-49fa-8112-48a1b95433b3 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:41.592760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 755/1503] 10.222.0.36 () {54 vars in 964 bytes} [Wed Aug 31 09:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:41.597613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32320ad-c102-414a-b078-86df9ae7dd5d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:41.597889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32320ad-c102-414a-b078-86df9ae7dd5d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:41.598163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32320ad-c102-414a-b078-86df9ae7dd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:41.629217 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:41.674534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-c32320ad-c102-414a-b078-86df9ae7dd5d None None] MFA Rules not processed for user `21d8ade66b7e4a1880c961c5e13adf04`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:41.675761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c32320ad-c102-414a-b078-86df9ae7dd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:41.752839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 749/1504] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Aug 31 09:44:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:41.763003 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febefc1-8ee6-49fa-8112-48a1b95433b3 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:41.764578 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-5febefc1-8ee6-49fa-8112-48a1b95433b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:41.814812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:41.872630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febefc1-8ee6-49fa-8112-48a1b95433b3 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:41.950508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ee790-607b-468f-9cb4-d3ba4d32f5d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:41.951139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ee790-607b-468f-9cb4-d3ba4d32f5d4 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:41.951435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ee790-607b-468f-9cb4-d3ba4d32f5d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:41.951712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ee790-607b-468f-9cb4-d3ba4d32f5d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:41.953344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ff2ee790-607b-468f-9cb4-d3ba4d32f5d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:42.052731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ee790-607b-468f-9cb4-d3ba4d32f5d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:42.054180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ee790-607b-468f-9cb4-d3ba4d32f5d4 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:42.058123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:42.073909 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 756/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:42.118709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289be6f-ffc1-404d-a908-1c1ca44b4c31 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:42.119184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289be6f-ffc1-404d-a908-1c1ca44b4c31 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:42.119401 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289be6f-ffc1-404d-a908-1c1ca44b4c31 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:42.120656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 750/1506] 10.222.0.36 () {52 vars in 830 bytes} [Wed Aug 31 09:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:42.124251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ee790-607b-468f-9cb4-d3ba4d32f5d4 service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:42.125448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ff2ee790-607b-468f-9cb4-d3ba4d32f5d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:42.225925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ee790-607b-468f-9cb4-d3ba4d32f5d4 service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:42.322031 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:42.322739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:42.323017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:42.323315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] PATH_INFO: `/v3/users/25ac0ff73f7543cfb86754dc3fb1c1cf` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:42.335567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=25ac0ff73f7543cfb86754dc3fb1c1cf)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:42.337095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: WARNING py.warnings [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:42.337095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: warnings.warn(msg) Aug 31 09:44:42.337095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: Aug 31 09:44:42.338297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:42.347769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:42.406507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25ac0ff73f7543cfb86754dc3fb1c1cf'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:42.429882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25ac0ff73f7543cfb86754dc3fb1c1cf'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:42.430930 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25ac0ff73f7543cfb86754dc3fb1c1cf'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:42.431513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25ac0ff73f7543cfb86754dc3fb1c1cf'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:42.442982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25ac0ff73f7543cfb86754dc3fb1c1cf'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:42.464984 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 757/1507] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:44:42 2022] DELETE /identity/v3/users/25ac0ff73f7543cfb86754dc3fb1c1cf => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:44:42.469297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e6ec0-a1c2-46a2-98ef-b5d4ef06b084 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:42.469575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e6ec0-a1c2-46a2-98ef-b5d4ef06b084 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:42.469845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e6ec0-a1c2-46a2-98ef-b5d4ef06b084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:42.528235 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-180e6ec0-a1c2-46a2-98ef-b5d4ef06b084 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:42.529385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-180e6ec0-a1c2-46a2-98ef-b5d4ef06b084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:42.605225 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:42.619230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 751/1508] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:42.628700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:42.629912 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:42.730698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dae2d6-f385-436b-9bc9-20b66da2f0ee admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:42.784899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:42.806393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e1eff-d8fd-4a29-a2b1-01e37f8e03e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:42.806919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e1eff-d8fd-4a29-a2b1-01e37f8e03e5 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:42.807224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e1eff-d8fd-4a29-a2b1-01e37f8e03e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:42.807498 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e1eff-d8fd-4a29-a2b1-01e37f8e03e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:42.809095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-632e1eff-d8fd-4a29-a2b1-01e37f8e03e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:42.920773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e1eff-d8fd-4a29-a2b1-01e37f8e03e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:42.922270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e1eff-d8fd-4a29-a2b1-01e37f8e03e5 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:42.941168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 758/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:43.307631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e6ec0-a1c2-46a2-98ef-b5d4ef06b084 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:43.308888 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-180e6ec0-a1c2-46a2-98ef-b5d4ef06b084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:43.320559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e6ec0-a1c2-46a2-98ef-b5d4ef06b084 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:43.418704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:43.419366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:43.419645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:43.419913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] PATH_INFO: `/v3/projects/165ec418946f4a9bb4d4d12d9cd25f67` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:43.430128 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] RBAC: Authorizing `identity:delete_project(project_id=165ec418946f4a9bb4d4d12d9cd25f67)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:43.431207 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: WARNING py.warnings [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:43.431207 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: warnings.warn(msg) Aug 31 09:44:43.431207 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: Aug 31 09:44:43.432428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:43.493882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '165ec418946f4a9bb4d4d12d9cd25f67', 'request_id': 'req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:43.507361 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '165ec418946f4a9bb4d4d12d9cd25f67', 'request_id': 'req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:43.514131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '165ec418946f4a9bb4d4d12d9cd25f67', 'request_id': 'req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:43.541096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] The token cache is being invalidate because project 165ec418946f4a9bb4d4d12d9cd25f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:44:43.541388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:43.541826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d59a76fa-ccc3-4e95-8d1a-c84f2156799a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:43.543270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 752/1510] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:44:43 2022] DELETE /identity/v3/projects/165ec418946f4a9bb4d4d12d9cd25f67 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:44:52.938859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=62 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:53.538843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:16.520215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d169b-b39e-4da6-8fdf-0163ff28b38e None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:16.520850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d169b-b39e-4da6-8fdf-0163ff28b38e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:16.520850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d169b-b39e-4da6-8fdf-0163ff28b38e None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:16.522108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:16.527534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8679e-8e81-4bad-a573-026c38eb9685 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:16.527995 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8679e-8e81-4bad-a573-026c38eb9685 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:16.528092 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8679e-8e81-4bad-a573-026c38eb9685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:16.613403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-d7e8679e-8e81-4bad-a573-026c38eb9685 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:16.614627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d7e8679e-8e81-4bad-a573-026c38eb9685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:16.842871 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:17.001841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d169b-b39e-4da6-8fdf-0163ff28b38e None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:17.004303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-763d169b-b39e-4da6-8fdf-0163ff28b38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:17.125133 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d169b-b39e-4da6-8fdf-0163ff28b38e None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:17.130586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3702da-d5b0-4efa-ac59-44870d7dc451 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:17.131478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3702da-d5b0-4efa-ac59-44870d7dc451 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:17.131758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3702da-d5b0-4efa-ac59-44870d7dc451 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:17.132033 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3702da-d5b0-4efa-ac59-44870d7dc451 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:17.140275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3702da-d5b0-4efa-ac59-44870d7dc451 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:17.141856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3702da-d5b0-4efa-ac59-44870d7dc451 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:17.151798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 760/1513] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:17.921517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3ea3c-02df-4432-9bca-f415a14b3cc8 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:17.922095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3ea3c-02df-4432-9bca-f415a14b3cc8 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:17.922095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3ea3c-02df-4432-9bca-f415a14b3cc8 None None] PATH_INFO: `/v3` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:17.923446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 754/1514] 10.222.0.71 () {54 vars in 925 bytes} [Wed Aug 31 09:46:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:17.931084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c04480-f7b5-4673-ba39-705e0fb9e05b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:17.932105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c04480-f7b5-4673-ba39-705e0fb9e05b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:17.932236 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c04480-f7b5-4673-ba39-705e0fb9e05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:18.000425 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3954781-70c1-47ea-a214-68462030a6d6 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:18.000794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3954781-70c1-47ea-a214-68462030a6d6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:18.001090 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3954781-70c1-47ea-a214-68462030a6d6 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:18.002167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-06c04480-f7b5-4673-ba39-705e0fb9e05b None None] MFA Rules not processed for user `560df3eb05f14b6489158ab053f282ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:18.002630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 755/1515] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:46:18.003790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-06c04480-f7b5-4673-ba39-705e0fb9e05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:18.007939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f458d-c7ae-4ee9-8ae1-e239f42d4b0b None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:18.008223 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f458d-c7ae-4ee9-8ae1-e239f42d4b0b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:18.008492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f458d-c7ae-4ee9-8ae1-e239f42d4b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:18.036591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-996f458d-c7ae-4ee9-8ae1-e239f42d4b0b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:18.037796 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-996f458d-c7ae-4ee9-8ae1-e239f42d4b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:18.046538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 756/1516] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:46:18.080493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:18.086470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 761/1517] 10.222.0.71 () {58 vars in 1015 bytes} [Wed Aug 31 09:46:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:46:18.201690 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f458d-c7ae-4ee9-8ae1-e239f42d4b0b None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:18.214219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f458d-c7ae-4ee9-8ae1-e239f42d4b0b None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:18.220015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f936f-a53a-4b93-a25a-b796d653919e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:18.220610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f936f-a53a-4b93-a25a-b796d653919e service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:18.220883 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f936f-a53a-4b93-a25a-b796d653919e service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:18.221151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f936f-a53a-4b93-a25a-b796d653919e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:18.229902 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f936f-a53a-4b93-a25a-b796d653919e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:18.231505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f936f-a53a-4b93-a25a-b796d653919e service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:18.241092 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 757/1518] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:27.421410 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b282f-1992-4b34-b7d7-d02e35dbe852 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:27.421973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b282f-1992-4b34-b7d7-d02e35dbe852 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:27.421973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b282f-1992-4b34-b7d7-d02e35dbe852 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:27.423260 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 762/1519] 10.222.0.36 () {54 vars in 961 bytes} [Wed Aug 31 09:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:27.428523 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d96a85-132c-4b5d-a462-4c18fad53fec None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:27.428995 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d96a85-132c-4b5d-a462-4c18fad53fec None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:27.429101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d96a85-132c-4b5d-a462-4c18fad53fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:27.448982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a8d96a85-132c-4b5d-a462-4c18fad53fec None None] MFA Rules not processed for user `560df3eb05f14b6489158ab053f282ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:27.450191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a8d96a85-132c-4b5d-a462-4c18fad53fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:27.458928 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 758/1520] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Aug 31 09:46:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:27.468177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b282f-1992-4b34-b7d7-d02e35dbe852 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:27.482492 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b282f-1992-4b34-b7d7-d02e35dbe852 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:27.487433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf34c87-87ce-462c-b9b5-da513b377e42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:27.488053 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf34c87-87ce-462c-b9b5-da513b377e42 service placement] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:27.488332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf34c87-87ce-462c-b9b5-da513b377e42 service placement] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:27.488602 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf34c87-87ce-462c-b9b5-da513b377e42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:27.497069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf34c87-87ce-462c-b9b5-da513b377e42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:27.498646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf34c87-87ce-462c-b9b5-da513b377e42 service placement] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:27.508557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 763/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:29.464553 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b669c4-db02-44cb-8c67-3687ce3eb376 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:29.465169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b669c4-db02-44cb-8c67-3687ce3eb376 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:29.465169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b669c4-db02-44cb-8c67-3687ce3eb376 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:29.466484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:46:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:29.471478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5a539-a0d1-4d93-99cb-c5bc4dc3b6f1 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:29.471938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5a539-a0d1-4d93-99cb-c5bc4dc3b6f1 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:29.472039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5a539-a0d1-4d93-99cb-c5bc4dc3b6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:29.492057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b0a5a539-a0d1-4d93-99cb-c5bc4dc3b6f1 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:29.493263 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b0a5a539-a0d1-4d93-99cb-c5bc4dc3b6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:29.501084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:29.657730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b669c4-db02-44cb-8c67-3687ce3eb376 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:29.672340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b669c4-db02-44cb-8c67-3687ce3eb376 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:29.677738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5f479-2ffd-445c-82a8-b73ab4808519 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:29.678323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5f479-2ffd-445c-82a8-b73ab4808519 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:29.678656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5f479-2ffd-445c-82a8-b73ab4808519 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:29.678858 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5f479-2ffd-445c-82a8-b73ab4808519 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:29.686965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5f479-2ffd-445c-82a8-b73ab4808519 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:29.688544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5f479-2ffd-445c-82a8-b73ab4808519 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:29.698890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 760/1524] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:30.527598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcc1bd-4e27-410d-9ec7-b0cb67a35937 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:30.528181 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcc1bd-4e27-410d-9ec7-b0cb67a35937 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:30.528181 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcc1bd-4e27-410d-9ec7-b0cb67a35937 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:30.529562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:30.534408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e95865-ac66-4a97-b94e-b62256c961b4 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:30.534943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e95865-ac66-4a97-b94e-b62256c961b4 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:30.534943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e95865-ac66-4a97-b94e-b62256c961b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:30.554419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-60e95865-ac66-4a97-b94e-b62256c961b4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:30.555630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-60e95865-ac66-4a97-b94e-b62256c961b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:30.563790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:30.718360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcc1bd-4e27-410d-9ec7-b0cb67a35937 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:30.732803 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcc1bd-4e27-410d-9ec7-b0cb67a35937 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:30.738244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bf340-94ca-458e-9242-e42604bbad3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:30.738835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bf340-94ca-458e-9242-e42604bbad3d service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:30.739112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bf340-94ca-458e-9242-e42604bbad3d service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:30.739412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bf340-94ca-458e-9242-e42604bbad3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:30.747860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bf340-94ca-458e-9242-e42604bbad3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:30.749424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bf340-94ca-458e-9242-e42604bbad3d service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:30.758943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 766/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:34.886115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345febe4-b1d6-4255-b8d1-7b31a8559096 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:34.886786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345febe4-b1d6-4255-b8d1-7b31a8559096 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:34.886786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345febe4-b1d6-4255-b8d1-7b31a8559096 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:34.888032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:34.893350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80452e6-be44-41b7-95ea-393796aadd6b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:34.893760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80452e6-be44-41b7-95ea-393796aadd6b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:34.893881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80452e6-be44-41b7-95ea-393796aadd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:34.914144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-a80452e6-be44-41b7-95ea-393796aadd6b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:34.915448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a80452e6-be44-41b7-95ea-393796aadd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:34.923358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:35.217127 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345febe4-b1d6-4255-b8d1-7b31a8559096 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:35.218965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-345febe4-b1d6-4255-b8d1-7b31a8559096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:35.246097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:35.299212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345febe4-b1d6-4255-b8d1-7b31a8559096 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:35.302724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96991d99-6b33-431c-a229-4f72d7a1ea9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:35.303364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96991d99-6b33-431c-a229-4f72d7a1ea9d service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:35.303639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96991d99-6b33-431c-a229-4f72d7a1ea9d service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:35.303905 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96991d99-6b33-431c-a229-4f72d7a1ea9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:35.313027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96991d99-6b33-431c-a229-4f72d7a1ea9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:35.314468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96991d99-6b33-431c-a229-4f72d7a1ea9d service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:35.323890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 763/1530] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:37.238392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453624d2-297d-4b41-9b2c-1793e32497a2 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:37.238961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453624d2-297d-4b41-9b2c-1793e32497a2 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:37.238961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453624d2-297d-4b41-9b2c-1793e32497a2 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:37.240324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:37.246415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c7055-0174-44df-83aa-89b5a4a8ed60 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:37.246866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c7055-0174-44df-83aa-89b5a4a8ed60 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:37.246963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c7055-0174-44df-83aa-89b5a4a8ed60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:37.268097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-d79c7055-0174-44df-83aa-89b5a4a8ed60 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:37.269306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d79c7055-0174-44df-83aa-89b5a4a8ed60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:37.277439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:37.563879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453624d2-297d-4b41-9b2c-1793e32497a2 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:37.574900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453624d2-297d-4b41-9b2c-1793e32497a2 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:37.578585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0fa48-2265-41b3-a002-e7aecd41c51c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:37.579205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0fa48-2265-41b3-a002-e7aecd41c51c service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:37.579490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0fa48-2265-41b3-a002-e7aecd41c51c service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:37.579765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0fa48-2265-41b3-a002-e7aecd41c51c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:37.589308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0fa48-2265-41b3-a002-e7aecd41c51c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:37.590755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0fa48-2265-41b3-a002-e7aecd41c51c service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:37.600404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 769/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:41.236295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0cea6-2f4e-403a-b719-adc0c14fc44e None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:41.236861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0cea6-2f4e-403a-b719-adc0c14fc44e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:41.236861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0cea6-2f4e-403a-b719-adc0c14fc44e None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:41.238176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:41.243322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007c9c0-2905-4cbb-b8e9-d127ec0b856f None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:41.243741 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007c9c0-2905-4cbb-b8e9-d127ec0b856f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:41.244054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007c9c0-2905-4cbb-b8e9-d127ec0b856f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:41.263447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-f007c9c0-2905-4cbb-b8e9-d127ec0b856f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:41.264658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-f007c9c0-2905-4cbb-b8e9-d127ec0b856f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:41.272423 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:41.831490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0cea6-2f4e-403a-b719-adc0c14fc44e None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:41.833264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-29e0cea6-2f4e-403a-b719-adc0c14fc44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:41.913246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0cea6-2f4e-403a-b719-adc0c14fc44e None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:41.916945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a414834-6bc4-4c5c-a72d-85b12c0ebc9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:41.917547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a414834-6bc4-4c5c-a72d-85b12c0ebc9c service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:41.917824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a414834-6bc4-4c5c-a72d-85b12c0ebc9c service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:41.918096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a414834-6bc4-4c5c-a72d-85b12c0ebc9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:41.926900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a414834-6bc4-4c5c-a72d-85b12c0ebc9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:41.928360 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a414834-6bc4-4c5c-a72d-85b12c0ebc9c service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:41.938289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 766/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:43.894154 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907271cc-271b-4af0-aa82-311504079160 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:43.894679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907271cc-271b-4af0-aa82-311504079160 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:43.894679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907271cc-271b-4af0-aa82-311504079160 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:43.896000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:43.901209 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29464bc0-df61-4cbd-b934-d743959ec007 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:43.901667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29464bc0-df61-4cbd-b934-d743959ec007 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:43.901786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29464bc0-df61-4cbd-b934-d743959ec007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:43.922972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-29464bc0-df61-4cbd-b934-d743959ec007 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:43.924197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-29464bc0-df61-4cbd-b934-d743959ec007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:43.932779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:44.498448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907271cc-271b-4af0-aa82-311504079160 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:44.500186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-907271cc-271b-4af0-aa82-311504079160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:44.513937 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907271cc-271b-4af0-aa82-311504079160 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:44.517514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bd862-0726-4f29-a8ff-2583efef385b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:44.518104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bd862-0726-4f29-a8ff-2583efef385b service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:44.518385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bd862-0726-4f29-a8ff-2583efef385b service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:44.518657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bd862-0726-4f29-a8ff-2583efef385b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:44.528105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bd862-0726-4f29-a8ff-2583efef385b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:44.529533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bd862-0726-4f29-a8ff-2583efef385b service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:44.538822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 772/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:53.918065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=66 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:54.534711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=34 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:13.865137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0da257-727d-4ce2-93d6-70dc51b08b76 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.873362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0da257-727d-4ce2-93d6-70dc51b08b76 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.874208 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0da257-727d-4ce2-93d6-70dc51b08b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.883181 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf8d7f-126f-4b77-be33-ea286e707ada None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.884756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf8d7f-126f-4b77-be33-ea286e707ada None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.885229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf8d7f-126f-4b77-be33-ea286e707ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.924468 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-d3bf8d7f-126f-4b77-be33-ea286e707ada None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:13.925793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d3bf8d7f-126f-4b77-be33-ea286e707ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.929503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-6a0da257-727d-4ce2-93d6-70dc51b08b76 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:13.932038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6a0da257-727d-4ce2-93d6-70dc51b08b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.948771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 773/1540] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:13.959756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2239eef-c447-46a4-a178-c8e3654a710a None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.960077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2239eef-c447-46a4-a178-c8e3654a710a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.960710 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2239eef-c447-46a4-a178-c8e3654a710a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.974889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 768/1541] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:13.995585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b2239eef-c447-46a4-a178-c8e3654a710a None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:13.996951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b2239eef-c447-46a4-a178-c8e3654a710a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.008545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854314c-038d-4ccd-904b-ba4e5db65325 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.009018 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854314c-038d-4ccd-904b-ba4e5db65325 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.009499 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854314c-038d-4ccd-904b-ba4e5db65325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.017413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 774/1542] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:14.022983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2239eef-c447-46a4-a178-c8e3654a710a None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.039297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2239eef-c447-46a4-a178-c8e3654a710a None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.044552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da507d0f-7ce2-4ecc-af81-275b82250908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.045209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da507d0f-7ce2-4ecc-af81-275b82250908 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.045494 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da507d0f-7ce2-4ecc-af81-275b82250908 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.045770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da507d0f-7ce2-4ecc-af81-275b82250908 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.046673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da507d0f-7ce2-4ecc-af81-275b82250908 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:14.046991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da507d0f-7ce2-4ecc-af81-275b82250908 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.048081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: WARNING py.warnings [None req-da507d0f-7ce2-4ecc-af81-275b82250908 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:48:14.048081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: warnings.warn(msg) Aug 31 09:48:14.048081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: Aug 31 09:48:14.055155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da507d0f-7ce2-4ecc-af81-275b82250908 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.075970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a854314c-038d-4ccd-904b-ba4e5db65325 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.077065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 775/1543] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:48:14.079161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a854314c-038d-4ccd-904b-ba4e5db65325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.088754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da507d0f-7ce2-4ecc-af81-275b82250908 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.103075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da507d0f-7ce2-4ecc-af81-275b82250908 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.114105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 769/1544] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:14.119844 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648a3e3-8d9e-4b96-8bca-a029e0c31328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.121178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648a3e3-8d9e-4b96-8bca-a029e0c31328 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.121847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648a3e3-8d9e-4b96-8bca-a029e0c31328 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.122615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648a3e3-8d9e-4b96-8bca-a029e0c31328 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.123670 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648a3e3-8d9e-4b96-8bca-a029e0c31328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.123823 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854314c-038d-4ccd-904b-ba4e5db65325 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.126221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648a3e3-8d9e-4b96-8bca-a029e0c31328 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.151010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854314c-038d-4ccd-904b-ba4e5db65325 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.169996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee2e86-83f9-436c-9a36-30d299171ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.170982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee2e86-83f9-436c-9a36-30d299171ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.175329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee2e86-83f9-436c-9a36-30d299171ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.175329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee2e86-83f9-436c-9a36-30d299171ba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.175329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee2e86-83f9-436c-9a36-30d299171ba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:14.175329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee2e86-83f9-436c-9a36-30d299171ba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.181093 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee2e86-83f9-436c-9a36-30d299171ba4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.212190 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 776/1545] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:14.213965 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648a3e3-8d9e-4b96-8bca-a029e0c31328 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.221373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 770/1546] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:14.242361 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee2e86-83f9-436c-9a36-30d299171ba4 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.268843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee2e86-83f9-436c-9a36-30d299171ba4 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.380632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648a3e3-8d9e-4b96-8bca-a029e0c31328 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.402912 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e87f48-0dcd-45ae-ae3c-4dfd2fa8533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.404551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e87f48-0dcd-45ae-ae3c-4dfd2fa8533e admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.405142 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e87f48-0dcd-45ae-ae3c-4dfd2fa8533e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.405705 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e87f48-0dcd-45ae-ae3c-4dfd2fa8533e admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.406950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e87f48-0dcd-45ae-ae3c-4dfd2fa8533e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.409144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: WARNING py.warnings [None req-e1e87f48-0dcd-45ae-ae3c-4dfd2fa8533e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:48:14.409144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: warnings.warn(msg) Aug 31 09:48:14.409144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: Aug 31 09:48:14.412283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e87f48-0dcd-45ae-ae3c-4dfd2fa8533e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.418677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a8693-4500-457b-86b7-63ca88cf763f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.419340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a8693-4500-457b-86b7-63ca88cf763f admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.419617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a8693-4500-457b-86b7-63ca88cf763f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.419887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a8693-4500-457b-86b7-63ca88cf763f admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.420746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a8693-4500-457b-86b7-63ca88cf763f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.421824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: WARNING py.warnings [None req-828a8693-4500-457b-86b7-63ca88cf763f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:48:14.421824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: warnings.warn(msg) Aug 31 09:48:14.421824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: Aug 31 09:48:14.423041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a8693-4500-457b-86b7-63ca88cf763f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.470597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 777/1547] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/users => generated 287 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:14.477782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833995ca-3430-48b7-8cef-4c0e211acd8b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.478044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833995ca-3430-48b7-8cef-4c0e211acd8b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.478667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833995ca-3430-48b7-8cef-4c0e211acd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.484837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 771/1548] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/projects => generated 391 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:14.486237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b88435-9a12-460f-b427-b3eb3d7d467e None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.486522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b88435-9a12-460f-b427-b3eb3d7d467e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.486795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b88435-9a12-460f-b427-b3eb3d7d467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.530016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-833995ca-3430-48b7-8cef-4c0e211acd8b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.532038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-833995ca-3430-48b7-8cef-4c0e211acd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.532869 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-d6b88435-9a12-460f-b427-b3eb3d7d467e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.534460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d6b88435-9a12-460f-b427-b3eb3d7d467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.652141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 778/1549] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:14.653393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833995ca-3430-48b7-8cef-4c0e211acd8b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.671165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833995ca-3430-48b7-8cef-4c0e211acd8b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.683186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5741b-107c-4359-8a04-302e710419ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.683186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5741b-107c-4359-8a04-302e710419ee admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.683186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5741b-107c-4359-8a04-302e710419ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.683186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5741b-107c-4359-8a04-302e710419ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.683757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5741b-107c-4359-8a04-302e710419ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.686884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5741b-107c-4359-8a04-302e710419ee admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.705544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 772/1550] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:14.706895 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b88435-9a12-460f-b427-b3eb3d7d467e None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.710999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 779/1551] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:14.710999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e87201-5e3b-4e77-8d2d-8356029735db None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.710999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e87201-5e3b-4e77-8d2d-8356029735db None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.711436 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e87201-5e3b-4e77-8d2d-8356029735db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.724010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b88435-9a12-460f-b427-b3eb3d7d467e None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.730065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f639570-2c03-46ad-9cbc-b39554aa5026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.730890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f639570-2c03-46ad-9cbc-b39554aa5026 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.731202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f639570-2c03-46ad-9cbc-b39554aa5026 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.731625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f639570-2c03-46ad-9cbc-b39554aa5026 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.732559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f639570-2c03-46ad-9cbc-b39554aa5026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.735099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f639570-2c03-46ad-9cbc-b39554aa5026 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.744058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-85e87201-5e3b-4e77-8d2d-8356029735db None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.746294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-85e87201-5e3b-4e77-8d2d-8356029735db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.768895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 780/1552] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:14.768895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bb750-5d4c-460a-9e6f-ae61328a720b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.768895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bb750-5d4c-460a-9e6f-ae61328a720b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.768895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bb750-5d4c-460a-9e6f-ae61328a720b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.775439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 773/1553] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/users => generated 296 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:14.777599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247aa04-fa81-4b9c-9664-724dc4868449 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.778330 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247aa04-fa81-4b9c-9664-724dc4868449 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.778765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247aa04-fa81-4b9c-9664-724dc4868449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.794415 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-a62bb750-5d4c-460a-9e6f-ae61328a720b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.795912 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a62bb750-5d4c-460a-9e6f-ae61328a720b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.804495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-7247aa04-fa81-4b9c-9664-724dc4868449 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.805749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7247aa04-fa81-4b9c-9664-724dc4868449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.806256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 781/1554] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:14.808124 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58278a2a-b0a1-4e2d-a718-23fe9d129a8f None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.808522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58278a2a-b0a1-4e2d-a718-23fe9d129a8f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.808903 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58278a2a-b0a1-4e2d-a718-23fe9d129a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.826148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 774/1555] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:14.828791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6a336-dee1-4cc2-ace9-ab58655bb296 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.829216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6a336-dee1-4cc2-ace9-ab58655bb296 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.829676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6a336-dee1-4cc2-ace9-ab58655bb296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.841697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-58278a2a-b0a1-4e2d-a718-23fe9d129a8f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.842970 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-58278a2a-b0a1-4e2d-a718-23fe9d129a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.857564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 782/1556] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:14.857564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58278a2a-b0a1-4e2d-a718-23fe9d129a8f None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.858372 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:14.858592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-50f6a336-dee1-4cc2-ace9-ab58655bb296 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.859898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-50f6a336-dee1-4cc2-ace9-ab58655bb296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.870056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 775/1557] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:14.871322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58278a2a-b0a1-4e2d-a718-23fe9d129a8f None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.871503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191f210-13bf-4e9d-ab9f-17bbccd83d4b None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.871611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191f210-13bf-4e9d-ab9f-17bbccd83d4b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.871884 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191f210-13bf-4e9d-ab9f-17bbccd83d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.880915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20fe2b-29ce-4a0f-be7d-a57165128ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.883605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20fe2b-29ce-4a0f-be7d-a57165128ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.884692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20fe2b-29ce-4a0f-be7d-a57165128ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.884692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20fe2b-29ce-4a0f-be7d-a57165128ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.885020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20fe2b-29ce-4a0f-be7d-a57165128ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.886785 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20fe2b-29ce-4a0f-be7d-a57165128ce9 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.898406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4191f210-13bf-4e9d-ab9f-17bbccd83d4b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.900007 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4191f210-13bf-4e9d-ab9f-17bbccd83d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.902259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 783/1558] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:14.904917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150db11-a3f3-45f0-835f-1e87793d5f13 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.905380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150db11-a3f3-45f0-835f-1e87793d5f13 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.905860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150db11-a3f3-45f0-835f-1e87793d5f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.908840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 776/1559] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:14.909949 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191f210-13bf-4e9d-ab9f-17bbccd83d4b None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.924449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191f210-13bf-4e9d-ab9f-17bbccd83d4b None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.930753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619ebae-af86-40a2-8e6f-c304f008dd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.931422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619ebae-af86-40a2-8e6f-c304f008dd13 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.931730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619ebae-af86-40a2-8e6f-c304f008dd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.932013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619ebae-af86-40a2-8e6f-c304f008dd13 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.932620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619ebae-af86-40a2-8e6f-c304f008dd13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.934343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619ebae-af86-40a2-8e6f-c304f008dd13 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.936269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-c150db11-a3f3-45f0-835f-1e87793d5f13 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.939479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c150db11-a3f3-45f0-835f-1e87793d5f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.947271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 777/1560] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:14.948698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b8fbc-ed0a-4dfc-89dd-a55472414d58 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.949121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b8fbc-ed0a-4dfc-89dd-a55472414d58 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.949395 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b8fbc-ed0a-4dfc-89dd-a55472414d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.956276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 784/1561] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:14.958863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b3bc4-ed02-4169-bd25-8e159c7624e4 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.959996 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b3bc4-ed02-4169-bd25-8e159c7624e4 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.961211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b3bc4-ed02-4169-bd25-8e159c7624e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.975740 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-802b8fbc-ed0a-4dfc-89dd-a55472414d58 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.977047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-802b8fbc-ed0a-4dfc-89dd-a55472414d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.985482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 778/1562] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:14.986365 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b8fbc-ed0a-4dfc-89dd-a55472414d58 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.995088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:14.997990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-542b3bc4-ed02-4169-bd25-8e159c7624e4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.999575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b8fbc-ed0a-4dfc-89dd-a55472414d58 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.000454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-542b3bc4-ed02-4169-bd25-8e159c7624e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:15.007863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee9547-d9a3-4cb3-92fc-1ddd1e3ea5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.008501 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee9547-d9a3-4cb3-92fc-1ddd1e3ea5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.008779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee9547-d9a3-4cb3-92fc-1ddd1e3ea5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.009050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee9547-d9a3-4cb3-92fc-1ddd1e3ea5d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.009721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee9547-d9a3-4cb3-92fc-1ddd1e3ea5d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:15.010020 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee9547-d9a3-4cb3-92fc-1ddd1e3ea5d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.011721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee9547-d9a3-4cb3-92fc-1ddd1e3ea5d7 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.021162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 785/1563] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:15.021162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5b355-f33c-4239-b34e-8e85f1d00777 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.021162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5b355-f33c-4239-b34e-8e85f1d00777 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.021162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5b355-f33c-4239-b34e-8e85f1d00777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.026206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 779/1564] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:15.027083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee9547-d9a3-4cb3-92fc-1ddd1e3ea5d7 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.037334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee9547-d9a3-4cb3-92fc-1ddd1e3ea5d7 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.042919 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf128d-9020-4478-a88a-e798e46e0b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.043635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf128d-9020-4478-a88a-e798e46e0b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.043916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf128d-9020-4478-a88a-e798e46e0b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.044194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf128d-9020-4478-a88a-e798e46e0b6f admin admin] PATH_INFO: `/v3/projects/04f9057993004696adf85b46b8cc2ff2/users/d85ee7e6a90f4e8a8bfd7da370140ad9/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.045958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e8f5b355-f33c-4239-b34e-8e85f1d00777 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:15.047250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e8f5b355-f33c-4239-b34e-8e85f1d00777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:15.057140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 786/1565] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:15.057758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5b355-f33c-4239-b34e-8e85f1d00777 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.071527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf128d-9020-4478-a88a-e798e46e0b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f9057993004696adf85b46b8cc2ff2, user_id=d85ee7e6a90f4e8a8bfd7da370140ad9, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.073344 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: WARNING py.warnings [None req-a8cf128d-9020-4478-a88a-e798e46e0b6f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:48:15.073344 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: warnings.warn(msg) Aug 31 09:48:15.073344 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: Aug 31 09:48:15.076115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf128d-9020-4478-a88a-e798e46e0b6f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.079734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5b355-f33c-4239-b34e-8e85f1d00777 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.087902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee52a7-da45-4149-9584-d5b532e6251f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.088858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee52a7-da45-4149-9584-d5b532e6251f admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.089266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee52a7-da45-4149-9584-d5b532e6251f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.089732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee52a7-da45-4149-9584-d5b532e6251f admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.090570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee52a7-da45-4149-9584-d5b532e6251f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:15.090998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee52a7-da45-4149-9584-d5b532e6251f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.093029 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee52a7-da45-4149-9584-d5b532e6251f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.093996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 780/1566] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:15 2022] PUT /identity/v3/projects/04f9057993004696adf85b46b8cc2ff2/users/d85ee7e6a90f4e8a8bfd7da370140ad9/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:15.100558 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf128d-9020-4478-a88a-e798e46e0b6f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.107585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 787/1567] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:15.108540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee52a7-da45-4149-9584-d5b532e6251f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.132884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee52a7-da45-4149-9584-d5b532e6251f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.205072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf128d-9020-4478-a88a-e798e46e0b6f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.210050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67948106-6c79-45b4-8e5d-35381ae56987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.210676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67948106-6c79-45b4-8e5d-35381ae56987 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.210956 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67948106-6c79-45b4-8e5d-35381ae56987 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.211269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67948106-6c79-45b4-8e5d-35381ae56987 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.211923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67948106-6c79-45b4-8e5d-35381ae56987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.212524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d9dfb-d9c1-4dca-a2c6-207fab929bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.213278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d9dfb-d9c1-4dca-a2c6-207fab929bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.213582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67948106-6c79-45b4-8e5d-35381ae56987 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.213800 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d9dfb-d9c1-4dca-a2c6-207fab929bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.214211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d9dfb-d9c1-4dca-a2c6-207fab929bb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.215084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d9dfb-d9c1-4dca-a2c6-207fab929bb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:15.215634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d9dfb-d9c1-4dca-a2c6-207fab929bb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.217719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d9dfb-d9c1-4dca-a2c6-207fab929bb8 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.226024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 781/1568] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:15.227622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67948106-6c79-45b4-8e5d-35381ae56987 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.237994 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 788/1569] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:15.239159 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d9dfb-d9c1-4dca-a2c6-207fab929bb8 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.241920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67948106-6c79-45b4-8e5d-35381ae56987 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.248332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6dcdb-46dd-4764-acaa-ae227ab75cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.249073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6dcdb-46dd-4764-acaa-ae227ab75cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.249475 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6dcdb-46dd-4764-acaa-ae227ab75cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.249925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6dcdb-46dd-4764-acaa-ae227ab75cd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.250695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6dcdb-46dd-4764-acaa-ae227ab75cd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:15.251094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6dcdb-46dd-4764-acaa-ae227ab75cd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.251537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d9dfb-d9c1-4dca-a2c6-207fab929bb8 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.253305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6dcdb-46dd-4764-acaa-ae227ab75cd4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.256909 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87408cba-2b2d-4186-8ef5-7ef50e2ce85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.257764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87408cba-2b2d-4186-8ef5-7ef50e2ce85f admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.258052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87408cba-2b2d-4186-8ef5-7ef50e2ce85f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.258330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87408cba-2b2d-4186-8ef5-7ef50e2ce85f admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.258994 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87408cba-2b2d-4186-8ef5-7ef50e2ce85f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:15.259332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87408cba-2b2d-4186-8ef5-7ef50e2ce85f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.261127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87408cba-2b2d-4186-8ef5-7ef50e2ce85f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.276532 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 782/1570] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:15.276532 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6dcdb-46dd-4764-acaa-ae227ab75cd4 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.281812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 789/1571] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:15.282628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87408cba-2b2d-4186-8ef5-7ef50e2ce85f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.283204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6dcdb-46dd-4764-acaa-ae227ab75cd4 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.289214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383a807-7995-446d-955f-ea5df9b39b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.289958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383a807-7995-446d-955f-ea5df9b39b35 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.290310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383a807-7995-446d-955f-ea5df9b39b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.290653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383a807-7995-446d-955f-ea5df9b39b35 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.291422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383a807-7995-446d-955f-ea5df9b39b35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.293624 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383a807-7995-446d-955f-ea5df9b39b35 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.295215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87408cba-2b2d-4186-8ef5-7ef50e2ce85f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.300355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507ca51-1f35-4040-9c0b-21ed8ec54e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.300976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507ca51-1f35-4040-9c0b-21ed8ec54e2b admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.301295 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507ca51-1f35-4040-9c0b-21ed8ec54e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.301591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507ca51-1f35-4040-9c0b-21ed8ec54e2b admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.302257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507ca51-1f35-4040-9c0b-21ed8ec54e2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:15.302555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507ca51-1f35-4040-9c0b-21ed8ec54e2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.304819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507ca51-1f35-4040-9c0b-21ed8ec54e2b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.322388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 790/1572] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:15.323757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21713269-ee3d-4efe-ac77-ae613451dfc5 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.324067 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21713269-ee3d-4efe-ac77-ae613451dfc5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.324396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21713269-ee3d-4efe-ac77-ae613451dfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.327677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 783/1573] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:15.328686 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383a807-7995-446d-955f-ea5df9b39b35 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.362051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-21713269-ee3d-4efe-ac77-ae613451dfc5 None None] MFA Rules not processed for user `d85ee7e6a90f4e8a8bfd7da370140ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:15.363348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-21713269-ee3d-4efe-ac77-ae613451dfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:15.424339 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383a807-7995-446d-955f-ea5df9b39b35 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.433266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21c89d-1da9-4097-a0c6-d215ab6c3977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.434173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21c89d-1da9-4097-a0c6-d215ab6c3977 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.434595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21c89d-1da9-4097-a0c6-d215ab6c3977 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.435061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21c89d-1da9-4097-a0c6-d215ab6c3977 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.435935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21c89d-1da9-4097-a0c6-d215ab6c3977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.438700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21c89d-1da9-4097-a0c6-d215ab6c3977 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.458516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 791/1574] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:15.460324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21713269-ee3d-4efe-ac77-ae613451dfc5 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.476299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21713269-ee3d-4efe-ac77-ae613451dfc5 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.478015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 784/1575] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:15.478929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21c89d-1da9-4097-a0c6-d215ab6c3977 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.495969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:15.496644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:15.554574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21c89d-1da9-4097-a0c6-d215ab6c3977 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.556945 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91a26e-acdb-47d7-be0f-8d395c5b9758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.557867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91a26e-acdb-47d7-be0f-8d395c5b9758 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.558175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91a26e-acdb-47d7-be0f-8d395c5b9758 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.558573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91a26e-acdb-47d7-be0f-8d395c5b9758 admin admin] PATH_INFO: `/v3/projects/bfef7bc61a644c12a42cc17b1aed1b31/users/b8818863a81b4238a22035fbb256568e/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.559724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831bb6b-0fb4-4bdf-97a8-0a4b4f6611f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.560363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831bb6b-0fb4-4bdf-97a8-0a4b4f6611f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.560640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831bb6b-0fb4-4bdf-97a8-0a4b4f6611f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.561143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831bb6b-0fb4-4bdf-97a8-0a4b4f6611f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.561562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831bb6b-0fb4-4bdf-97a8-0a4b4f6611f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.563591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831bb6b-0fb4-4bdf-97a8-0a4b4f6611f2 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.580343 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91a26e-acdb-47d7-be0f-8d395c5b9758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfef7bc61a644c12a42cc17b1aed1b31, user_id=b8818863a81b4238a22035fbb256568e, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.585076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91a26e-acdb-47d7-be0f-8d395c5b9758 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.598424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 785/1576] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:15.599617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831bb6b-0fb4-4bdf-97a8-0a4b4f6611f2 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.605725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 792/1577] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:15 2022] PUT /identity/v3/projects/bfef7bc61a644c12a42cc17b1aed1b31/users/b8818863a81b4238a22035fbb256568e/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:15.607178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91a26e-acdb-47d7-be0f-8d395c5b9758 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.679156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831bb6b-0fb4-4bdf-97a8-0a4b4f6611f2 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.684873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf09c05-1d84-447f-95e7-36dfc679db8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.685712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf09c05-1d84-447f-95e7-36dfc679db8e admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.686054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf09c05-1d84-447f-95e7-36dfc679db8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.686378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf09c05-1d84-447f-95e7-36dfc679db8e admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.687728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf09c05-1d84-447f-95e7-36dfc679db8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.690158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf09c05-1d84-447f-95e7-36dfc679db8e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.691639 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91a26e-acdb-47d7-be0f-8d395c5b9758 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.697611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83415f4-a2c1-4340-86f0-ae80e9bcf145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.699006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83415f4-a2c1-4340-86f0-ae80e9bcf145 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.699360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83415f4-a2c1-4340-86f0-ae80e9bcf145 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.699595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83415f4-a2c1-4340-86f0-ae80e9bcf145 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.700255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83415f4-a2c1-4340-86f0-ae80e9bcf145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.702212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83415f4-a2c1-4340-86f0-ae80e9bcf145 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.725487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 786/1578] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:15.726289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf09c05-1d84-447f-95e7-36dfc679db8e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.737031 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 793/1579] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:15.738808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83415f4-a2c1-4340-86f0-ae80e9bcf145 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.806879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf09c05-1d84-447f-95e7-36dfc679db8e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.812920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da4618-2ca1-4f3b-aede-cbcc49f05f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.813664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da4618-2ca1-4f3b-aede-cbcc49f05f6b admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.814137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da4618-2ca1-4f3b-aede-cbcc49f05f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.814455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da4618-2ca1-4f3b-aede-cbcc49f05f6b admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.815364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da4618-2ca1-4f3b-aede-cbcc49f05f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.815518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83415f4-a2c1-4340-86f0-ae80e9bcf145 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.817823 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da4618-2ca1-4f3b-aede-cbcc49f05f6b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.820813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120ecec-b349-4f66-b5df-09c09397584c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.821447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120ecec-b349-4f66-b5df-09c09397584c admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.821727 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120ecec-b349-4f66-b5df-09c09397584c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.822007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120ecec-b349-4f66-b5df-09c09397584c admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.822663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120ecec-b349-4f66-b5df-09c09397584c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.824783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120ecec-b349-4f66-b5df-09c09397584c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.847189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 787/1580] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:15.848355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da4618-2ca1-4f3b-aede-cbcc49f05f6b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.861521 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da4618-2ca1-4f3b-aede-cbcc49f05f6b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.864003 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 794/1581] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:15.864937 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120ecec-b349-4f66-b5df-09c09397584c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.871140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0f406-e6f5-4a58-8614-c7994423f450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.871890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0f406-e6f5-4a58-8614-c7994423f450 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.872034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0f406-e6f5-4a58-8614-c7994423f450 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.872346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0f406-e6f5-4a58-8614-c7994423f450 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.873002 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0f406-e6f5-4a58-8614-c7994423f450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.874990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0f406-e6f5-4a58-8614-c7994423f450 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.878333 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:15.891263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:15.919601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 788/1582] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:15.919601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0f406-e6f5-4a58-8614-c7994423f450 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.944475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120ecec-b349-4f66-b5df-09c09397584c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.993707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0f406-e6f5-4a58-8614-c7994423f450 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.998656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d08ed-0891-4ea8-ae7e-3c906dcf23fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.999489 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d08ed-0891-4ea8-ae7e-3c906dcf23fb admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.999941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d08ed-0891-4ea8-ae7e-3c906dcf23fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.000232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d08ed-0891-4ea8-ae7e-3c906dcf23fb admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.000963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d08ed-0891-4ea8-ae7e-3c906dcf23fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.002963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d08ed-0891-4ea8-ae7e-3c906dcf23fb admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.028275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015307a4-e961-4784-b0e3-295b278a1256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.028903 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015307a4-e961-4784-b0e3-295b278a1256 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.029185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015307a4-e961-4784-b0e3-295b278a1256 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.029459 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015307a4-e961-4784-b0e3-295b278a1256 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.030129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015307a4-e961-4784-b0e3-295b278a1256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.032134 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015307a4-e961-4784-b0e3-295b278a1256 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.033430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 789/1583] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:16.035545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a7ad7-4d0d-48a3-b077-57e77eafef5d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.036013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a7ad7-4d0d-48a3-b077-57e77eafef5d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.036482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a7ad7-4d0d-48a3-b077-57e77eafef5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.057933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 795/1584] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:16.058244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015307a4-e961-4784-b0e3-295b278a1256 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.064528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f62a7ad7-4d0d-48a3-b077-57e77eafef5d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:16.065770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f62a7ad7-4d0d-48a3-b077-57e77eafef5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:16.074699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 790/1585] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:16.075378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a7ad7-4d0d-48a3-b077-57e77eafef5d None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.085121 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015307a4-e961-4784-b0e3-295b278a1256 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.086725 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a7ad7-4d0d-48a3-b077-57e77eafef5d None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.092684 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cb39c-fad0-4552-8e58-0ab38703dd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.093414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cb39c-fad0-4552-8e58-0ab38703dd31 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.093882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cb39c-fad0-4552-8e58-0ab38703dd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.094365 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cb39c-fad0-4552-8e58-0ab38703dd31 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.094975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cb39c-fad0-4552-8e58-0ab38703dd31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.096180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd06aa2-9502-4090-b86c-d92fec95045c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.097212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cb39c-fad0-4552-8e58-0ab38703dd31 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.097446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd06aa2-9502-4090-b86c-d92fec95045c admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.097898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd06aa2-9502-4090-b86c-d92fec95045c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.098485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd06aa2-9502-4090-b86c-d92fec95045c admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.100302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd06aa2-9502-4090-b86c-d92fec95045c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.104689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd06aa2-9502-4090-b86c-d92fec95045c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.128095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 791/1586] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/users => generated 288 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:16.129802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cb39c-fad0-4552-8e58-0ab38703dd31 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.138091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 796/1587] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/users => generated 286 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:16.142286 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd06aa2-9502-4090-b86c-d92fec95045c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.151373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cb39c-fad0-4552-8e58-0ab38703dd31 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.153952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd06aa2-9502-4090-b86c-d92fec95045c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.158826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a3cdc-6ef4-4c84-8849-3f470ef8c3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.159294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc985a-de59-42bd-86c6-f948d5bf55f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.159487 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a3cdc-6ef4-4c84-8849-3f470ef8c3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.159769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a3cdc-6ef4-4c84-8849-3f470ef8c3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.160042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a3cdc-6ef4-4c84-8849-3f470ef8c3d6 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.160342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc985a-de59-42bd-86c6-f948d5bf55f7 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.160692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a3cdc-6ef4-4c84-8849-3f470ef8c3d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.160827 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc985a-de59-42bd-86c6-f948d5bf55f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.161285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc985a-de59-42bd-86c6-f948d5bf55f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.162500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc985a-de59-42bd-86c6-f948d5bf55f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.162839 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a3cdc-6ef4-4c84-8849-3f470ef8c3d6 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.165654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc985a-de59-42bd-86c6-f948d5bf55f7 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.183311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 792/1588] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.184204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc985a-de59-42bd-86c6-f948d5bf55f7 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.195448 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc985a-de59-42bd-86c6-f948d5bf55f7 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.200264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927962a-adc1-4a1c-b5e4-c9acb6ebe49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.200880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927962a-adc1-4a1c-b5e4-c9acb6ebe49b admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.201484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927962a-adc1-4a1c-b5e4-c9acb6ebe49b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.201627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 797/1589] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/users => generated 288 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:16.201726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927962a-adc1-4a1c-b5e4-c9acb6ebe49b admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.202067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927962a-adc1-4a1c-b5e4-c9acb6ebe49b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.202871 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a3cdc-6ef4-4c84-8849-3f470ef8c3d6 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.204152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927962a-adc1-4a1c-b5e4-c9acb6ebe49b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.220845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a3cdc-6ef4-4c84-8849-3f470ef8c3d6 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.234915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 793/1590] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/users => generated 293 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:16.234915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927962a-adc1-4a1c-b5e4-c9acb6ebe49b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.235946 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a71d56-d9a2-4733-a1ad-3f38b0d1667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.236728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a71d56-d9a2-4733-a1ad-3f38b0d1667f admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.237131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a71d56-d9a2-4733-a1ad-3f38b0d1667f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.237791 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a71d56-d9a2-4733-a1ad-3f38b0d1667f admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.239174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a71d56-d9a2-4733-a1ad-3f38b0d1667f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.244654 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a71d56-d9a2-4733-a1ad-3f38b0d1667f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.246574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927962a-adc1-4a1c-b5e4-c9acb6ebe49b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.252437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b40f26-bf75-4888-8f8e-14f6689d7b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.253239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b40f26-bf75-4888-8f8e-14f6689d7b9a admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.253680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b40f26-bf75-4888-8f8e-14f6689d7b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.254307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b40f26-bf75-4888-8f8e-14f6689d7b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.255143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b40f26-bf75-4888-8f8e-14f6689d7b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.257340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b40f26-bf75-4888-8f8e-14f6689d7b9a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.259348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 798/1591] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.260830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a71d56-d9a2-4733-a1ad-3f38b0d1667f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.275700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 794/1592] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.275700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b40f26-bf75-4888-8f8e-14f6689d7b9a admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.284990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b40f26-bf75-4888-8f8e-14f6689d7b9a admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.291502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcf473-768f-43bc-b848-353e31aedde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.292349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcf473-768f-43bc-b848-353e31aedde3 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.292689 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcf473-768f-43bc-b848-353e31aedde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.293156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcf473-768f-43bc-b848-353e31aedde3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.293889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcf473-768f-43bc-b848-353e31aedde3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.296625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcf473-768f-43bc-b848-353e31aedde3 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.315841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 795/1593] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.317589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcf473-768f-43bc-b848-353e31aedde3 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.332673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcf473-768f-43bc-b848-353e31aedde3 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.339115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04261ff1-b0d0-464f-8408-844325832723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.339934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04261ff1-b0d0-464f-8408-844325832723 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.340398 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04261ff1-b0d0-464f-8408-844325832723 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.340858 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04261ff1-b0d0-464f-8408-844325832723 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.341669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04261ff1-b0d0-464f-8408-844325832723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.343852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04261ff1-b0d0-464f-8408-844325832723 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.348613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a71d56-d9a2-4733-a1ad-3f38b0d1667f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.354674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b1161-5b98-404f-9b34-15361776243d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.355682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b1161-5b98-404f-9b34-15361776243d service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.356071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b1161-5b98-404f-9b34-15361776243d service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.356526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b1161-5b98-404f-9b34-15361776243d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.359076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 796/1594] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.363765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04261ff1-b0d0-464f-8408-844325832723 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.370734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b1161-5b98-404f-9b34-15361776243d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.373602 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b1161-5b98-404f-9b34-15361776243d service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.377687 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04261ff1-b0d0-464f-8408-844325832723 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.383578 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e18ca-9c86-4ced-9001-c319ab7c368e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.384376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e18ca-9c86-4ced-9001-c319ab7c368e admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.384773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e18ca-9c86-4ced-9001-c319ab7c368e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.385240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e18ca-9c86-4ced-9001-c319ab7c368e admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.386148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e18ca-9c86-4ced-9001-c319ab7c368e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.388195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e18ca-9c86-4ced-9001-c319ab7c368e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.394246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 799/1595] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:16.396120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b1161-5b98-404f-9b34-15361776243d service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.409100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 797/1596] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.409100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e18ca-9c86-4ced-9001-c319ab7c368e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.411831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:16.419980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b1161-5b98-404f-9b34-15361776243d service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.428064 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e18ca-9c86-4ced-9001-c319ab7c368e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.431552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e1eeb-d875-474a-ad5a-5ddb29c9a0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.432708 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e1eeb-d875-474a-ad5a-5ddb29c9a0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.433223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e1eeb-d875-474a-ad5a-5ddb29c9a0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.433687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e1eeb-d875-474a-ad5a-5ddb29c9a0d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.434880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e1eeb-d875-474a-ad5a-5ddb29c9a0d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.437449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9998e-b9ae-4bed-a657-82e753bf40be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.437665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e1eeb-d875-474a-ad5a-5ddb29c9a0d8 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.438625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9998e-b9ae-4bed-a657-82e753bf40be admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.439346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9998e-b9ae-4bed-a657-82e753bf40be admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.439795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9998e-b9ae-4bed-a657-82e753bf40be admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.441130 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9998e-b9ae-4bed-a657-82e753bf40be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.444327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9998e-b9ae-4bed-a657-82e753bf40be admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.463316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 800/1597] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.463316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e1eeb-d875-474a-ad5a-5ddb29c9a0d8 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.465131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 798/1598] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.466873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9998e-b9ae-4bed-a657-82e753bf40be admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.487362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9998e-b9ae-4bed-a657-82e753bf40be admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.487845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e1eeb-d875-474a-ad5a-5ddb29c9a0d8 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.496304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4af6ff-6daa-4d8f-859f-576bec2ab716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.497447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2b772-2fd4-4c6c-bb5e-c728a5415335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.497670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4af6ff-6daa-4d8f-859f-576bec2ab716 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.498077 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4af6ff-6daa-4d8f-859f-576bec2ab716 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.498778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2b772-2fd4-4c6c-bb5e-c728a5415335 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.498919 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4af6ff-6daa-4d8f-859f-576bec2ab716 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.499314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2b772-2fd4-4c6c-bb5e-c728a5415335 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.499836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2b772-2fd4-4c6c-bb5e-c728a5415335 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.500302 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4af6ff-6daa-4d8f-859f-576bec2ab716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.501023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2b772-2fd4-4c6c-bb5e-c728a5415335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.502388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4af6ff-6daa-4d8f-859f-576bec2ab716 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.504365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2b772-2fd4-4c6c-bb5e-c728a5415335 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.521306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 799/1599] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.523526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4af6ff-6daa-4d8f-859f-576bec2ab716 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.545145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 801/1600] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.545431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4af6ff-6daa-4d8f-859f-576bec2ab716 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.545625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2b772-2fd4-4c6c-bb5e-c728a5415335 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.554687 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9f582-d116-4585-a97c-bb807e11d7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.555742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9f582-d116-4585-a97c-bb807e11d7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.556190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9f582-d116-4585-a97c-bb807e11d7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.557035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9f582-d116-4585-a97c-bb807e11d7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.558507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9f582-d116-4585-a97c-bb807e11d7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.561998 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9f582-d116-4585-a97c-bb807e11d7f8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.583715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2b772-2fd4-4c6c-bb5e-c728a5415335 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.585186 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 800/1601] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.586346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9f582-d116-4585-a97c-bb807e11d7f8 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.599323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bb2d6-12f4-4de2-a94d-ba7d4627f47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.601386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bb2d6-12f4-4de2-a94d-ba7d4627f47f admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.602147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bb2d6-12f4-4de2-a94d-ba7d4627f47f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.602556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bb2d6-12f4-4de2-a94d-ba7d4627f47f admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.603908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bb2d6-12f4-4de2-a94d-ba7d4627f47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.607554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bb2d6-12f4-4de2-a94d-ba7d4627f47f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.608648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9f582-d116-4585-a97c-bb807e11d7f8 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.617709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b304dc7-dfba-4019-bd22-cdfe8c2e9ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.618660 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b304dc7-dfba-4019-bd22-cdfe8c2e9ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.619095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b304dc7-dfba-4019-bd22-cdfe8c2e9ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.619563 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b304dc7-dfba-4019-bd22-cdfe8c2e9ed6 admin admin] PATH_INFO: `/v3/projects/09f25e83fff24a8e924bb22e6f9fdd46/users/8a3df25e740c4c7f8a311d5a1574c3ba/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.636079 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 802/1602] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.636079 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bb2d6-12f4-4de2-a94d-ba7d4627f47f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.639012 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:16.644476 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b304dc7-dfba-4019-bd22-cdfe8c2e9ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f25e83fff24a8e924bb22e6f9fdd46, user_id=8a3df25e740c4c7f8a311d5a1574c3ba, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.649147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bb2d6-12f4-4de2-a94d-ba7d4627f47f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.650001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b304dc7-dfba-4019-bd22-cdfe8c2e9ed6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.654809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed40a92-a510-4d78-bce1-bae39245a854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.655906 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed40a92-a510-4d78-bce1-bae39245a854 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.656580 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed40a92-a510-4d78-bce1-bae39245a854 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.656580 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed40a92-a510-4d78-bce1-bae39245a854 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.657309 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed40a92-a510-4d78-bce1-bae39245a854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.659301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed40a92-a510-4d78-bce1-bae39245a854 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.674077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 803/1604] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.674399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 801/1604] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:16 2022] PUT /identity/v3/projects/09f25e83fff24a8e924bb22e6f9fdd46/users/8a3df25e740c4c7f8a311d5a1574c3ba/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:16.675023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed40a92-a510-4d78-bce1-bae39245a854 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.675641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b304dc7-dfba-4019-bd22-cdfe8c2e9ed6 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.686744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed40a92-a510-4d78-bce1-bae39245a854 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.795428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284cc1a-9dc8-4c81-9c05-7cbdccaeccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.796975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284cc1a-9dc8-4c81-9c05-7cbdccaeccdb admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.797277 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284cc1a-9dc8-4c81-9c05-7cbdccaeccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.797522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284cc1a-9dc8-4c81-9c05-7cbdccaeccdb admin admin] PATH_INFO: `/v3/projects/c5ba332e27424d1bb34a7b79734090f3/users/fcf76cf19b1547dca933307b675213d9/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.820389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b304dc7-dfba-4019-bd22-cdfe8c2e9ed6 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.821287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284cc1a-9dc8-4c81-9c05-7cbdccaeccdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ba332e27424d1bb34a7b79734090f3, user_id=fcf76cf19b1547dca933307b675213d9, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.825262 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284cc1a-9dc8-4c81-9c05-7cbdccaeccdb admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.826469 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb87850-5c88-4af0-b921-66438f4da626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.827532 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb87850-5c88-4af0-b921-66438f4da626 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.827777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb87850-5c88-4af0-b921-66438f4da626 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.828055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb87850-5c88-4af0-b921-66438f4da626 admin admin] PATH_INFO: `/v3/projects/c4914faafb494e5bb323114853f25c54/users/980ab18eb5c646378d738079621a0dbc/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.854143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 804/1605] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:16 2022] PUT /identity/v3/projects/c5ba332e27424d1bb34a7b79734090f3/users/fcf76cf19b1547dca933307b675213d9/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:16.854143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284cc1a-9dc8-4c81-9c05-7cbdccaeccdb admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.857539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb87850-5c88-4af0-b921-66438f4da626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4914faafb494e5bb323114853f25c54, user_id=980ab18eb5c646378d738079621a0dbc, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.863726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb87850-5c88-4af0-b921-66438f4da626 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.894736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 802/1606] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:16 2022] PUT /identity/v3/projects/c4914faafb494e5bb323114853f25c54/users/980ab18eb5c646378d738079621a0dbc/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:16.900372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb87850-5c88-4af0-b921-66438f4da626 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.971651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284cc1a-9dc8-4c81-9c05-7cbdccaeccdb admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.995762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb87850-5c88-4af0-b921-66438f4da626 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.006854 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf59816-2b83-499e-a4c8-8f3114d18943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.008591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf59816-2b83-499e-a4c8-8f3114d18943 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.009084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf59816-2b83-499e-a4c8-8f3114d18943 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.009635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf59816-2b83-499e-a4c8-8f3114d18943 admin admin] PATH_INFO: `/v3/projects/e45b8d5877454f1fadf465070b4a1694/users/43aa3506f7f64ea7a6d57f0087092334/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.044233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf59816-2b83-499e-a4c8-8f3114d18943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8d5877454f1fadf465070b4a1694, user_id=43aa3506f7f64ea7a6d57f0087092334, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.047584 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf59816-2b83-499e-a4c8-8f3114d18943 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.069878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:17.071279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 803/1607] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:16 2022] PUT /identity/v3/projects/e45b8d5877454f1fadf465070b4a1694/users/43aa3506f7f64ea7a6d57f0087092334/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:17.072561 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf59816-2b83-499e-a4c8-8f3114d18943 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.095904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bae5e-c3b6-4ceb-89c4-dabb13e74ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.096606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bae5e-c3b6-4ceb-89c4-dabb13e74ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.096914 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bae5e-c3b6-4ceb-89c4-dabb13e74ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.097187 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bae5e-c3b6-4ceb-89c4-dabb13e74ab2 admin admin] PATH_INFO: `/v3/projects/e5b56cc9afe641ae8f21b59f77301f1e/users/9ba257f98cb64603a82b1c702659bbcd/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.114947 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493bae5e-c3b6-4ceb-89c4-dabb13e74ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5b56cc9afe641ae8f21b59f77301f1e, user_id=9ba257f98cb64603a82b1c702659bbcd, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.118846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493bae5e-c3b6-4ceb-89c4-dabb13e74ab2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.140107 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 805/1608] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:16 2022] PUT /identity/v3/projects/e5b56cc9afe641ae8f21b59f77301f1e/users/9ba257f98cb64603a82b1c702659bbcd/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:17.148519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bae5e-c3b6-4ceb-89c4-dabb13e74ab2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.207338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:17.222873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf59816-2b83-499e-a4c8-8f3114d18943 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.248417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bae5e-c3b6-4ceb-89c4-dabb13e74ab2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.258290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99560ef-bf7e-4d53-befd-e4974861995b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.259001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99560ef-bf7e-4d53-befd-e4974861995b admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.259305 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99560ef-bf7e-4d53-befd-e4974861995b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.259583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99560ef-bf7e-4d53-befd-e4974861995b admin admin] PATH_INFO: `/v3/projects/568beb0822e640c1bacd98a2aec950ff/users/b6ca4276e1554d4184451313b2e09a88/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.279961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99560ef-bf7e-4d53-befd-e4974861995b admin admin] RBAC: Authorizing `identity:create_grant(project_id=568beb0822e640c1bacd98a2aec950ff, user_id=b6ca4276e1554d4184451313b2e09a88, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.282860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99560ef-bf7e-4d53-befd-e4974861995b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.308536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 806/1609] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:17 2022] PUT /identity/v3/projects/568beb0822e640c1bacd98a2aec950ff/users/b6ca4276e1554d4184451313b2e09a88/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:17.308536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cea620-20da-4cd1-a3c0-e3f2a517b79c None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.308536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cea620-20da-4cd1-a3c0-e3f2a517b79c None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.308536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cea620-20da-4cd1-a3c0-e3f2a517b79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.317689 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edbdd6-4aa6-42d1-af40-dd667cbdd108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.319438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edbdd6-4aa6-42d1-af40-dd667cbdd108 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.319571 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edbdd6-4aa6-42d1-af40-dd667cbdd108 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.319813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edbdd6-4aa6-42d1-af40-dd667cbdd108 admin admin] PATH_INFO: `/v3/projects/3ef3ed4828bc4d04bf1896d4b627dad5/users/39de236db3f84712a7982697d7ac2ad1/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.338292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59edbdd6-4aa6-42d1-af40-dd667cbdd108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ef3ed4828bc4d04bf1896d4b627dad5, user_id=39de236db3f84712a7982697d7ac2ad1, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.338592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-96cea620-20da-4cd1-a3c0-e3f2a517b79c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:17.340811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-96cea620-20da-4cd1-a3c0-e3f2a517b79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:17.341447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59edbdd6-4aa6-42d1-af40-dd667cbdd108 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.362397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 804/1610] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:17 2022] PUT /identity/v3/projects/3ef3ed4828bc4d04bf1896d4b627dad5/users/39de236db3f84712a7982697d7ac2ad1/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:17.363110 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edbdd6-4aa6-42d1-af40-dd667cbdd108 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.444978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edbdd6-4aa6-42d1-af40-dd667cbdd108 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.449806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0fa8e-5916-43e6-ba59-e5d7f90ad022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.450437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0fa8e-5916-43e6-ba59-e5d7f90ad022 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.450715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0fa8e-5916-43e6-ba59-e5d7f90ad022 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.451003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0fa8e-5916-43e6-ba59-e5d7f90ad022 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.451685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0fa8e-5916-43e6-ba59-e5d7f90ad022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.453320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0fa8e-5916-43e6-ba59-e5d7f90ad022 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.459976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 807/1611] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:17.462180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4526b3d-ec7c-47f2-b109-3a0a803c2496 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.462675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4526b3d-ec7c-47f2-b109-3a0a803c2496 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.472632 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 805/1612] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:17.473300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4526b3d-ec7c-47f2-b109-3a0a803c2496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.477746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dceaf-c983-416a-b6fa-cfbb771dd72f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.478233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dceaf-c983-416a-b6fa-cfbb771dd72f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.478662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dceaf-c983-416a-b6fa-cfbb771dd72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.503901 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b4526b3d-ec7c-47f2-b109-3a0a803c2496 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:17.506234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b4526b3d-ec7c-47f2-b109-3a0a803c2496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:17.515176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 808/1613] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:17.516008 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be07cb8-842d-43c6-bde1-54393690d86a None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.516494 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be07cb8-842d-43c6-bde1-54393690d86a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.516846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be07cb8-842d-43c6-bde1-54393690d86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.526450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-0c1dceaf-c983-416a-b6fa-cfbb771dd72f None None] MFA Rules not processed for user `43aa3506f7f64ea7a6d57f0087092334`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:17.528620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0c1dceaf-c983-416a-b6fa-cfbb771dd72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:17.560594 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-2be07cb8-842d-43c6-bde1-54393690d86a None None] MFA Rules not processed for user `9ba257f98cb64603a82b1c702659bbcd`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:17.562843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2be07cb8-842d-43c6-bde1-54393690d86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:17.641558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:17.697718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 809/1614] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:17.699026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14474b64-e4ae-46ce-ad5a-1097bf2f9459 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.699366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14474b64-e4ae-46ce-ad5a-1097bf2f9459 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.699645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14474b64-e4ae-46ce-ad5a-1097bf2f9459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.710381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 806/1615] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:17.712736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481f8a8-cdbb-40c6-b213-41ebdbd8179c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.713305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481f8a8-cdbb-40c6-b213-41ebdbd8179c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.713703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481f8a8-cdbb-40c6-b213-41ebdbd8179c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.770181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:17.787511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-14474b64-e4ae-46ce-ad5a-1097bf2f9459 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:17.789845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-0481f8a8-cdbb-40c6-b213-41ebdbd8179c None None] MFA Rules not processed for user `39de236db3f84712a7982697d7ac2ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:17.791708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0481f8a8-cdbb-40c6-b213-41ebdbd8179c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:17.792980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-14474b64-e4ae-46ce-ad5a-1097bf2f9459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:17.831472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 810/1616] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:17.835536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14474b64-e4ae-46ce-ad5a-1097bf2f9459 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.885510 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 807/1617] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:17.886853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481f8a8-cdbb-40c6-b213-41ebdbd8179c None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.903160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481f8a8-cdbb-40c6-b213-41ebdbd8179c None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.909477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3544b-59fc-4ccf-9ba5-71f63267ed77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.910873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3544b-59fc-4ccf-9ba5-71f63267ed77 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.911646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3544b-59fc-4ccf-9ba5-71f63267ed77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.911918 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3544b-59fc-4ccf-9ba5-71f63267ed77 admin admin] PATH_INFO: `/v3/projects/c5ba332e27424d1bb34a7b79734090f3/users/fcf76cf19b1547dca933307b675213d9/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.919723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3544b-59fc-4ccf-9ba5-71f63267ed77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ba332e27424d1bb34a7b79734090f3, user_id=fcf76cf19b1547dca933307b675213d9, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.927354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3544b-59fc-4ccf-9ba5-71f63267ed77 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.943720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14474b64-e4ae-46ce-ad5a-1097bf2f9459 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.956067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 808/1618] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:17 2022] PUT /identity/v3/projects/c5ba332e27424d1bb34a7b79734090f3/users/fcf76cf19b1547dca933307b675213d9/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:17.960607 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3544b-59fc-4ccf-9ba5-71f63267ed77 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:18.014800 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab286d7-1615-4acb-bcba-270db498c6f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.015547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab286d7-1615-4acb-bcba-270db498c6f1 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.015752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab286d7-1615-4acb-bcba-270db498c6f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.016032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab286d7-1615-4acb-bcba-270db498c6f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.109819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3544b-59fc-4ccf-9ba5-71f63267ed77 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:18.114500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7244ffc-a637-46aa-baf5-731c28934636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.116104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7244ffc-a637-46aa-baf5-731c28934636 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.116606 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7244ffc-a637-46aa-baf5-731c28934636 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.117065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7244ffc-a637-46aa-baf5-731c28934636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.202749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab286d7-1615-4acb-bcba-270db498c6f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.202976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7244ffc-a637-46aa-baf5-731c28934636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.204626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7244ffc-a637-46aa-baf5-731c28934636 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:18.206550 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab286d7-1615-4acb-bcba-270db498c6f1 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:18.221355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 809/1619] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:18.224074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9fba7-4c86-4861-bf1f-92defe64cc74 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.225854 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9fba7-4c86-4861-bf1f-92defe64cc74 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.226460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9fba7-4c86-4861-bf1f-92defe64cc74 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.229790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 810/1620] 10.222.0.36 () {54 vars in 962 bytes} [Wed Aug 31 09:48:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:18.232115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4ba05-14b8-4155-9e76-6fb43e7eb2c5 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.232661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4ba05-14b8-4155-9e76-6fb43e7eb2c5 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.233150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4ba05-14b8-4155-9e76-6fb43e7eb2c5 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.233269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 811/1621] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:18.234152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab286d7-1615-4acb-bcba-270db498c6f1 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:18.235853 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3ab286d7-1615-4acb-bcba-270db498c6f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:18.236616 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 811/1622] 10.222.0.36 () {54 vars in 962 bytes} [Wed Aug 31 09:48:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:18.238925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535761e9-a292-4c5f-a963-869613235446 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.239454 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535761e9-a292-4c5f-a963-869613235446 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.239975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535761e9-a292-4c5f-a963-869613235446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.250678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab286d7-1615-4acb-bcba-270db498c6f1 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:18.256792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f20af-14be-450d-8d58-8df49180054a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.258437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f20af-14be-450d-8d58-8df49180054a service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.258921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f20af-14be-450d-8d58-8df49180054a service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.259385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f20af-14be-450d-8d58-8df49180054a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.271398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f20af-14be-450d-8d58-8df49180054a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.274549 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f20af-14be-450d-8d58-8df49180054a service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:18.289851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 812/1623] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:18.292099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cf85a-a2fb-4d34-b281-4b35d562e235 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.292664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cf85a-a2fb-4d34-b281-4b35d562e235 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.294390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cf85a-a2fb-4d34-b281-4b35d562e235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.329741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-535761e9-a292-4c5f-a963-869613235446 None None] MFA Rules not processed for user `fcf76cf19b1547dca933307b675213d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:18.331401 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-535761e9-a292-4c5f-a963-869613235446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:18.349260 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-8e6cf85a-a2fb-4d34-b281-4b35d562e235 None None] MFA Rules not processed for user `ef72bb95992c4a21ae2dbb2a08c51421`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:18.352438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-8e6cf85a-a2fb-4d34-b281-4b35d562e235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:18.359417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:18.411041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:18.518168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 813/1624] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Aug 31 09:48:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:18.522426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67167e0b-08f3-4d36-af38-8149548910d4 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.522892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67167e0b-08f3-4d36-af38-8149548910d4 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.524589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67167e0b-08f3-4d36-af38-8149548910d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.567473 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 812/1625] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:18.577588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-67167e0b-08f3-4d36-af38-8149548910d4 None None] MFA Rules not processed for user `ef72bb95992c4a21ae2dbb2a08c51421`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:18.578841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535761e9-a292-4c5f-a963-869613235446 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:18.580395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-67167e0b-08f3-4d36-af38-8149548910d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:18.616093 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535761e9-a292-4c5f-a963-869613235446 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:18.620156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 814/1626] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Aug 31 09:48:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:18.623861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67167e0b-08f3-4d36-af38-8149548910d4 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:18.651077 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b615c-b014-41dc-b9f9-6fb8a84865ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.654585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b615c-b014-41dc-b9f9-6fb8a84865ff service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.654869 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b615c-b014-41dc-b9f9-6fb8a84865ff service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.655315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b615c-b014-41dc-b9f9-6fb8a84865ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.769123 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b615c-b014-41dc-b9f9-6fb8a84865ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.772281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b615c-b014-41dc-b9f9-6fb8a84865ff service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:18.800306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 813/1627] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:18.800306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b615c-b014-41dc-b9f9-6fb8a84865ff service nova] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:18.833538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b615c-b014-41dc-b9f9-6fb8a84865ff service nova] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:18.848121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95acca67-35fa-4bd6-a5c0-5b2107ce9520 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.861214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acca67-35fa-4bd6-a5c0-5b2107ce9520 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.861214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acca67-35fa-4bd6-a5c0-5b2107ce9520 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.861751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acca67-35fa-4bd6-a5c0-5b2107ce9520 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.887850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67167e0b-08f3-4d36-af38-8149548910d4 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:18.907625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90272c6b-6ea1-4060-ad66-2ba51cef3431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.915898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90272c6b-6ea1-4060-ad66-2ba51cef3431 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.916396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90272c6b-6ea1-4060-ad66-2ba51cef3431 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.917274 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90272c6b-6ea1-4060-ad66-2ba51cef3431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.958191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90272c6b-6ea1-4060-ad66-2ba51cef3431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.962466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90272c6b-6ea1-4060-ad66-2ba51cef3431 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:18.982196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95acca67-35fa-4bd6-a5c0-5b2107ce9520 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.984757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95acca67-35fa-4bd6-a5c0-5b2107ce9520 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:18.999765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 814/1628] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:19.020577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 815/1629] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:19.220632 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95acca67-35fa-4bd6-a5c0-5b2107ce9520 service nova] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.235014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95acca67-35fa-4bd6-a5c0-5b2107ce9520 service nova] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.327368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace728c7-c293-44d0-9abe-5e2ad02001f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.327368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace728c7-c293-44d0-9abe-5e2ad02001f5 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.327368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace728c7-c293-44d0-9abe-5e2ad02001f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.327368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace728c7-c293-44d0-9abe-5e2ad02001f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.397375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90272c6b-6ea1-4060-ad66-2ba51cef3431 service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.418729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90272c6b-6ea1-4060-ad66-2ba51cef3431 service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.431232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc3328-4089-4819-8bda-06253d1633b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.432641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc3328-4089-4819-8bda-06253d1633b8 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.433915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc3328-4089-4819-8bda-06253d1633b8 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.436182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc3328-4089-4819-8bda-06253d1633b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.522414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:19.526501 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace728c7-c293-44d0-9abe-5e2ad02001f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:19.528273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace728c7-c293-44d0-9abe-5e2ad02001f5 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.543495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 815/1630] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:19.639875 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc3328-4089-4819-8bda-06253d1633b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:19.643941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc3328-4089-4819-8bda-06253d1633b8 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.676590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 816/1631] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:19.786518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c6166-834a-4fc9-b931-3c593fc68871 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.787513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c6166-834a-4fc9-b931-3c593fc68871 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.789068 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c6166-834a-4fc9-b931-3c593fc68871 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.792495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 816/1632] 10.222.0.36 () {54 vars in 912 bytes} [Wed Aug 31 09:48:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:19.809456 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1fd6f-dd97-438f-a98e-7f817d4a91b8 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.809842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1fd6f-dd97-438f-a98e-7f817d4a91b8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.810380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1fd6f-dd97-438f-a98e-7f817d4a91b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.883972 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-cbe1fd6f-dd97-438f-a98e-7f817d4a91b8 None None] MFA Rules not processed for user `ef72bb95992c4a21ae2dbb2a08c51421`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:19.887595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-cbe1fd6f-dd97-438f-a98e-7f817d4a91b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:19.915731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 817/1633] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:48:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:19.928244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c6166-834a-4fc9-b931-3c593fc68871 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.957366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c6166-834a-4fc9-b931-3c593fc68871 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.968546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a47d2-52d5-437c-bb06-1a6dc5bd42da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.970550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a47d2-52d5-437c-bb06-1a6dc5bd42da service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.971257 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a47d2-52d5-437c-bb06-1a6dc5bd42da service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.973735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a47d2-52d5-437c-bb06-1a6dc5bd42da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.981178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576408-94c4-4de4-b392-705d318909f7 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.985133 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576408-94c4-4de4-b392-705d318909f7 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.986843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576408-94c4-4de4-b392-705d318909f7 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.990314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 818/1634] 10.222.0.36 () {54 vars in 912 bytes} [Wed Aug 31 09:48:19 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:20.012605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5812b9b-33dd-401a-b6a4-869e7a25266b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.013860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5812b9b-33dd-401a-b6a4-869e7a25266b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.014600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5812b9b-33dd-401a-b6a4-869e7a25266b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.052128 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a47d2-52d5-437c-bb06-1a6dc5bd42da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.054099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a47d2-52d5-437c-bb06-1a6dc5bd42da service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.073158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e5812b9b-33dd-401a-b6a4-869e7a25266b None None] MFA Rules not processed for user `ef72bb95992c4a21ae2dbb2a08c51421`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:20.079330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e5812b9b-33dd-401a-b6a4-869e7a25266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:20.123646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 817/1635] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:20.135401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 819/1636] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:48:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:20.185495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a47d2-52d5-437c-bb06-1a6dc5bd42da service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.208036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a47d2-52d5-437c-bb06-1a6dc5bd42da service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.230399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942fa67-9900-49ec-831b-350224110c86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.231035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942fa67-9900-49ec-831b-350224110c86 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.231358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942fa67-9900-49ec-831b-350224110c86 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.231627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942fa67-9900-49ec-831b-350224110c86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.249608 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942fa67-9900-49ec-831b-350224110c86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.251233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942fa67-9900-49ec-831b-350224110c86 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.277014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 818/1637] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:23.229742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5812b9b-33dd-401a-b6a4-869e7a25266b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:23.261526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5812b9b-33dd-401a-b6a4-869e7a25266b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:23.277242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:23.277744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:23.278246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:23.279717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] PATH_INFO: `/v3/users/43aa3506f7f64ea7a6d57f0087092334` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:23.283257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43aa3506f7f64ea7a6d57f0087092334)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:23.287521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:23.340451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:23.419763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43aa3506f7f64ea7a6d57f0087092334'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:23.457799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43aa3506f7f64ea7a6d57f0087092334'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:23.460108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43aa3506f7f64ea7a6d57f0087092334'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:23.461565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43aa3506f7f64ea7a6d57f0087092334'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:23.503262 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43aa3506f7f64ea7a6d57f0087092334'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:23.554809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 820/1638] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:23 2022] DELETE /identity/v3/users/43aa3506f7f64ea7a6d57f0087092334 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:23.561593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582dd55-409f-4557-90b0-2cda4ea0d112 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:23.564192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582dd55-409f-4557-90b0-2cda4ea0d112 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:23.565050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582dd55-409f-4557-90b0-2cda4ea0d112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:23.681240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a582dd55-409f-4557-90b0-2cda4ea0d112 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:23.685774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a582dd55-409f-4557-90b0-2cda4ea0d112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:23.856344 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 819/1639] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:23.925451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:23.934664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:24.174209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd49347-2803-45fd-9570-eb00def206e6 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:24.190289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10331f9-72d5-42af-9f1f-8c72d341f273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:24.191397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10331f9-72d5-42af-9f1f-8c72d341f273 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:24.243322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10331f9-72d5-42af-9f1f-8c72d341f273 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:24.243907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10331f9-72d5-42af-9f1f-8c72d341f273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:24.279898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10331f9-72d5-42af-9f1f-8c72d341f273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:24.282490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10331f9-72d5-42af-9f1f-8c72d341f273 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:24.316734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 821/1640] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:24.354029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582dd55-409f-4557-90b0-2cda4ea0d112 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:24.356776 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a582dd55-409f-4557-90b0-2cda4ea0d112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:24.417066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582dd55-409f-4557-90b0-2cda4ea0d112 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:24.441980 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:24.448869 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:24.449352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:24.449712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] PATH_INFO: `/v3/users/39de236db3f84712a7982697d7ac2ad1` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:24.451930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] RBAC: Authorizing `identity:delete_user(user_id=39de236db3f84712a7982697d7ac2ad1)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:24.458723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: WARNING py.warnings [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:48:24.458723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: warnings.warn(msg) Aug 31 09:48:24.458723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: Aug 31 09:48:24.464882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:24.691779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39de236db3f84712a7982697d7ac2ad1'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:24.722841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39de236db3f84712a7982697d7ac2ad1'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:24.725534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39de236db3f84712a7982697d7ac2ad1'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:24.842602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39de236db3f84712a7982697d7ac2ad1'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:24.843422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39de236db3f84712a7982697d7ac2ad1'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:24.891244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 820/1641] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:24 2022] DELETE /identity/v3/users/39de236db3f84712a7982697d7ac2ad1 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:24.896466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18a68c-7acc-4e68-8699-4364594c2df6 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:24.896896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18a68c-7acc-4e68-8699-4364594c2df6 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:24.897193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18a68c-7acc-4e68-8699-4364594c2df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:24.951957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0f18a68c-7acc-4e68-8699-4364594c2df6 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:24.954101 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0f18a68c-7acc-4e68-8699-4364594c2df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:25.262968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 822/1642] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:25.282224 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:25.284383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:25.447894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:25.486800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487500e-c791-4cae-b160-86392181d64e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:25.495654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d95611-2512-490d-a3de-0ada3a89076c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:25.496728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d95611-2512-490d-a3de-0ada3a89076c service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:25.497281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d95611-2512-490d-a3de-0ada3a89076c service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:25.497810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d95611-2512-490d-a3de-0ada3a89076c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:25.524227 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d95611-2512-490d-a3de-0ada3a89076c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:25.527705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d95611-2512-490d-a3de-0ada3a89076c service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:25.569649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 821/1643] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:26.588551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18a68c-7acc-4e68-8699-4364594c2df6 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:26.592673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0f18a68c-7acc-4e68-8699-4364594c2df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:26.656433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18a68c-7acc-4e68-8699-4364594c2df6 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:26.685441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:26.700063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.700407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.700695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] PATH_INFO: `/v3/projects/e45b8d5877454f1fadf465070b4a1694` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.719348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e45b8d5877454f1fadf465070b4a1694)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:26.721140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: WARNING py.warnings [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:48:26.721140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: warnings.warn(msg) Aug 31 09:48:26.721140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: Aug 31 09:48:26.721809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:26.837299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:26.893492 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e45b8d5877454f1fadf465070b4a1694', 'request_id': 'req-805ed270-cb8b-42c9-80be-d1b53217d91d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:26.930840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e45b8d5877454f1fadf465070b4a1694', 'request_id': 'req-805ed270-cb8b-42c9-80be-d1b53217d91d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:26.976479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e45b8d5877454f1fadf465070b4a1694', 'request_id': 'req-805ed270-cb8b-42c9-80be-d1b53217d91d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:27.009405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] The token cache is being invalidate because project e45b8d5877454f1fadf465070b4a1694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:27.009782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:27.011317 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-805ed270-cb8b-42c9-80be-d1b53217d91d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:27.015184 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 823/1644] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:26 2022] DELETE /identity/v3/projects/e45b8d5877454f1fadf465070b4a1694 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:27.020026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d029ab1-350c-4311-832f-62f528758e97 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.020577 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d029ab1-350c-4311-832f-62f528758e97 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.020577 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d029ab1-350c-4311-832f-62f528758e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.081186 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-0d029ab1-350c-4311-832f-62f528758e97 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:27.083536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0d029ab1-350c-4311-832f-62f528758e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:27.307580 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 822/1645] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:27.321024 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82397c58-8cdd-447c-8835-ca9ee191526d None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.321349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82397c58-8cdd-447c-8835-ca9ee191526d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.321603 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82397c58-8cdd-447c-8835-ca9ee191526d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.375602 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-82397c58-8cdd-447c-8835-ca9ee191526d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:27.377003 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-82397c58-8cdd-447c-8835-ca9ee191526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:27.413389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 824/1646] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:27.415815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d029ab1-350c-4311-832f-62f528758e97 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:27.432069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d029ab1-350c-4311-832f-62f528758e97 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:27.445742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb313b70-045d-4272-9d7a-f1d582e25be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:27.446388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb313b70-045d-4272-9d7a-f1d582e25be9 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.446667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb313b70-045d-4272-9d7a-f1d582e25be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.446942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb313b70-045d-4272-9d7a-f1d582e25be9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.447640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb313b70-045d-4272-9d7a-f1d582e25be9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:27.447949 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb313b70-045d-4272-9d7a-f1d582e25be9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:27.452411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb313b70-045d-4272-9d7a-f1d582e25be9 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:27.511325 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 823/1647] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:48:27.512863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82397c58-8cdd-447c-8835-ca9ee191526d None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:27.531259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82397c58-8cdd-447c-8835-ca9ee191526d None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:27.544191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0aa4c-eab2-4236-88eb-9925baaedaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:27.544845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0aa4c-eab2-4236-88eb-9925baaedaf2 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.545171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0aa4c-eab2-4236-88eb-9925baaedaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.545452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0aa4c-eab2-4236-88eb-9925baaedaf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.546173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0aa4c-eab2-4236-88eb-9925baaedaf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:27.548273 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0aa4c-eab2-4236-88eb-9925baaedaf2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:27.629557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 825/1648] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:27 2022] POST /identity/v3/projects => generated 405 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:27.637057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb313b70-045d-4272-9d7a-f1d582e25be9 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:27.679946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb313b70-045d-4272-9d7a-f1d582e25be9 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:27.958244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed28407-1119-4b3e-b34f-abff56ef135b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:27.959800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed28407-1119-4b3e-b34f-abff56ef135b admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.960296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed28407-1119-4b3e-b34f-abff56ef135b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.960783 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed28407-1119-4b3e-b34f-abff56ef135b admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.961975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed28407-1119-4b3e-b34f-abff56ef135b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:27.965831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed28407-1119-4b3e-b34f-abff56ef135b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:28.044049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0aa4c-eab2-4236-88eb-9925baaedaf2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:28.053654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 824/1649] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:27 2022] POST /identity/v3/users => generated 303 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:28.066437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0aa4c-eab2-4236-88eb-9925baaedaf2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:28.092668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0da96c-570d-4c43-8514-85b25184f24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:28.093329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0da96c-570d-4c43-8514-85b25184f24e admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:28.093613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0da96c-570d-4c43-8514-85b25184f24e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:28.093924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0da96c-570d-4c43-8514-85b25184f24e admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:28.098136 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0da96c-570d-4c43-8514-85b25184f24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:28.100615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0da96c-570d-4c43-8514-85b25184f24e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:28.124185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 826/1650] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:28.130379 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed28407-1119-4b3e-b34f-abff56ef135b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:28.161359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed28407-1119-4b3e-b34f-abff56ef135b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:28.181489 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa79dd-fb54-478e-aa13-8258c87abadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:28.182138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa79dd-fb54-478e-aa13-8258c87abadf admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:28.182610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa79dd-fb54-478e-aa13-8258c87abadf admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:28.183014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa79dd-fb54-478e-aa13-8258c87abadf admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:28.183866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa79dd-fb54-478e-aa13-8258c87abadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:28.186590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa79dd-fb54-478e-aa13-8258c87abadf admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:28.218009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0da96c-570d-4c43-8514-85b25184f24e admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:28.218704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 825/1651] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:28.233582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0da96c-570d-4c43-8514-85b25184f24e admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:28.245278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17170f09-ac7e-4eec-997e-4a53303af037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:28.246381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17170f09-ac7e-4eec-997e-4a53303af037 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:28.246900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17170f09-ac7e-4eec-997e-4a53303af037 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:28.247506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17170f09-ac7e-4eec-997e-4a53303af037 admin admin] PATH_INFO: `/v3/projects/e6653107d0a44ddfb81af0035d6b0640/users/4f6699455479413f8ab282d3f5dd3377/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:28.287586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17170f09-ac7e-4eec-997e-4a53303af037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6653107d0a44ddfb81af0035d6b0640, user_id=4f6699455479413f8ab282d3f5dd3377, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:28.296774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17170f09-ac7e-4eec-997e-4a53303af037 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:28.336155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 827/1652] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:28 2022] PUT /identity/v3/projects/e6653107d0a44ddfb81af0035d6b0640/users/4f6699455479413f8ab282d3f5dd3377/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:28.348070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fb92d-3184-4919-b5cb-0fb4bdc38909 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:28.348355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fb92d-3184-4919-b5cb-0fb4bdc38909 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:28.348795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fb92d-3184-4919-b5cb-0fb4bdc38909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:28.399093 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-9c9fb92d-3184-4919-b5cb-0fb4bdc38909 None None] MFA Rules not processed for user `4f6699455479413f8ab282d3f5dd3377`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:28.403904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9c9fb92d-3184-4919-b5cb-0fb4bdc38909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:28.571826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:28.607086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17170f09-ac7e-4eec-997e-4a53303af037 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:28.612251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-17170f09-ac7e-4eec-997e-4a53303af037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:28.735924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 826/1653] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:28 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:28.737853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9fb92d-3184-4919-b5cb-0fb4bdc38909 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:28.907509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17170f09-ac7e-4eec-997e-4a53303af037 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:28.915422 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:28.916111 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:28.916393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:28.916672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c admin admin] PATH_INFO: `/v3/projects/3ef3ed4828bc4d04bf1896d4b627dad5` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:28.919437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ef3ed4828bc4d04bf1896d4b627dad5)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:28.923923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:29.020162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9fb92d-3184-4919-b5cb-0fb4bdc38909 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:29.061705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d27dee-3a59-42f1-91b1-db8b137acbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:29.062945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d27dee-3a59-42f1-91b1-db8b137acbb8 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.063687 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d27dee-3a59-42f1-91b1-db8b137acbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.064653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d27dee-3a59-42f1-91b1-db8b137acbb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.066241 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d27dee-3a59-42f1-91b1-db8b137acbb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:29.083301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d27dee-3a59-42f1-91b1-db8b137acbb8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:29.102181 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ef3ed4828bc4d04bf1896d4b627dad5', 'request_id': 'req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.133189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ef3ed4828bc4d04bf1896d4b627dad5', 'request_id': 'req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.150433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ef3ed4828bc4d04bf1896d4b627dad5', 'request_id': 'req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.152719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:29.185948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c admin admin] The token cache is being invalidate because project 3ef3ed4828bc4d04bf1896d4b627dad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:29.186296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.194722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 827/1654] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:28 2022] POST /identity/v3/projects => generated 405 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:29.195174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5ecec90e-fe97-4105-ade1-1f59b31f4e9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.199415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d27dee-3a59-42f1-91b1-db8b137acbb8 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:29.203030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-06d27dee-3a59-42f1-91b1-db8b137acbb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:29.207279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 828/1655] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:28 2022] DELETE /identity/v3/projects/3ef3ed4828bc4d04bf1896d4b627dad5 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:29.212353 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee5f5c-56c4-4fbc-bb25-9c28afbd2efa None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.216262 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee5f5c-56c4-4fbc-bb25-9c28afbd2efa None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.216778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee5f5c-56c4-4fbc-bb25-9c28afbd2efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.292289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-99ee5f5c-56c4-4fbc-bb25-9c28afbd2efa None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:29.293660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-99ee5f5c-56c4-4fbc-bb25-9c28afbd2efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:29.469614 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d27dee-3a59-42f1-91b1-db8b137acbb8 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:29.486495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a27b8-3f40-48bc-a579-6085cae09aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:29.487151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a27b8-3f40-48bc-a579-6085cae09aac admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.487432 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a27b8-3f40-48bc-a579-6085cae09aac admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.487705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a27b8-3f40-48bc-a579-6085cae09aac admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.488358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a27b8-3f40-48bc-a579-6085cae09aac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:29.490424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a27b8-3f40-48bc-a579-6085cae09aac admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:29.534680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 828/1656] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:29 2022] POST /identity/v3/users => generated 309 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:29.534870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 829/1657] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:29.535176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd859125-8a2c-4cbf-a7d6-9ebaa084ddb1 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.535439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd859125-8a2c-4cbf-a7d6-9ebaa084ddb1 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.535708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd859125-8a2c-4cbf-a7d6-9ebaa084ddb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.536636 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee5f5c-56c4-4fbc-bb25-9c28afbd2efa None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:29.556135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee5f5c-56c4-4fbc-bb25-9c28afbd2efa None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:29.567881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10982851-18bb-4584-bf0f-f14ec2ce785c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:29.569258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10982851-18bb-4584-bf0f-f14ec2ce785c admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.570024 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10982851-18bb-4584-bf0f-f14ec2ce785c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.570140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10982851-18bb-4584-bf0f-f14ec2ce785c admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.571387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10982851-18bb-4584-bf0f-f14ec2ce785c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:29.574636 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10982851-18bb-4584-bf0f-f14ec2ce785c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:29.607896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 830/1658] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:29.611942 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10982851-18bb-4584-bf0f-f14ec2ce785c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:29.628160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-bd859125-8a2c-4cbf-a7d6-9ebaa084ddb1 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:29.629719 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-bd859125-8a2c-4cbf-a7d6-9ebaa084ddb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:29.638624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10982851-18bb-4584-bf0f-f14ec2ce785c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:29.660075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f4f09-2d79-4447-b179-acca8f3e2d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:29.660854 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f4f09-2d79-4447-b179-acca8f3e2d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.661115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f4f09-2d79-4447-b179-acca8f3e2d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.661391 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f4f09-2d79-4447-b179-acca8f3e2d05 admin admin] PATH_INFO: `/v3/projects/76cf84eb16374a3d92db67eb3186d394/users/5630dc664041438c95af9b555e4e91f9/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.671664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 829/1659] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:29.677382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd859125-8a2c-4cbf-a7d6-9ebaa084ddb1 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:29.701677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd859125-8a2c-4cbf-a7d6-9ebaa084ddb1 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:29.721109 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f4f09-2d79-4447-b179-acca8f3e2d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76cf84eb16374a3d92db67eb3186d394, user_id=5630dc664041438c95af9b555e4e91f9, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:29.722867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f4f09-2d79-4447-b179-acca8f3e2d05 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:29.726832 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050ab72-616f-4bfd-9243-fe682ba7b7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:29.727487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050ab72-616f-4bfd-9243-fe682ba7b7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.727761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050ab72-616f-4bfd-9243-fe682ba7b7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.728041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050ab72-616f-4bfd-9243-fe682ba7b7d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.728691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050ab72-616f-4bfd-9243-fe682ba7b7d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:29.728989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050ab72-616f-4bfd-9243-fe682ba7b7d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:29.730641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050ab72-616f-4bfd-9243-fe682ba7b7d8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:29.759289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 831/1660] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:29 2022] PUT /identity/v3/projects/76cf84eb16374a3d92db67eb3186d394/users/5630dc664041438c95af9b555e4e91f9/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:29.771190 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f4f09-2d79-4447-b179-acca8f3e2d05 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:29.781927 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 830/1661] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:48:29.782918 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050ab72-616f-4bfd-9243-fe682ba7b7d8 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:29.823891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050ab72-616f-4bfd-9243-fe682ba7b7d8 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:29.937163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:30.001519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f4f09-2d79-4447-b179-acca8f3e2d05 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:30.018095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34194d51-db9a-45c0-867a-20ab3e99df67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.018738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34194d51-db9a-45c0-867a-20ab3e99df67 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.019016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34194d51-db9a-45c0-867a-20ab3e99df67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.019323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34194d51-db9a-45c0-867a-20ab3e99df67 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.019989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34194d51-db9a-45c0-867a-20ab3e99df67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:30.021763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34194d51-db9a-45c0-867a-20ab3e99df67 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:30.024950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fa3d5-2fc1-4e48-932f-797d512db17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.025588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fa3d5-2fc1-4e48-932f-797d512db17e admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.025861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fa3d5-2fc1-4e48-932f-797d512db17e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.026129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fa3d5-2fc1-4e48-932f-797d512db17e admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.026784 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fa3d5-2fc1-4e48-932f-797d512db17e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:30.028805 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fa3d5-2fc1-4e48-932f-797d512db17e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:30.066049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 832/1662] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:30.069499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34194d51-db9a-45c0-867a-20ab3e99df67 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:30.105227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34194d51-db9a-45c0-867a-20ab3e99df67 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:30.122041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cceb6fc-3a46-48f1-9271-19c0db7170ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.122742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cceb6fc-3a46-48f1-9271-19c0db7170ae admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.123019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cceb6fc-3a46-48f1-9271-19c0db7170ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.123333 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cceb6fc-3a46-48f1-9271-19c0db7170ae admin admin] PATH_INFO: `/v3/projects/76cf84eb16374a3d92db67eb3186d394/users/5630dc664041438c95af9b555e4e91f9/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.129533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 831/1663] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:29 2022] POST /identity/v3/projects => generated 377 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:30.136321 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cceb6fc-3a46-48f1-9271-19c0db7170ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=76cf84eb16374a3d92db67eb3186d394, user_id=5630dc664041438c95af9b555e4e91f9, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:30.138445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fa3d5-2fc1-4e48-932f-797d512db17e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:30.144923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cceb6fc-3a46-48f1-9271-19c0db7170ae admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:30.215800 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 833/1664] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:30 2022] PUT /identity/v3/projects/76cf84eb16374a3d92db67eb3186d394/users/5630dc664041438c95af9b555e4e91f9/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:30.227983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68beadb2-e230-45d1-ad35-c1708818a377 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.228348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68beadb2-e230-45d1-ad35-c1708818a377 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.246703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68beadb2-e230-45d1-ad35-c1708818a377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.384739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-68beadb2-e230-45d1-ad35-c1708818a377 None None] MFA Rules not processed for user `5630dc664041438c95af9b555e4e91f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:30.386138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-68beadb2-e230-45d1-ad35-c1708818a377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:30.501309 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:30.620732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fa3d5-2fc1-4e48-932f-797d512db17e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:30.857174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76cceb-4e6a-489b-878f-aec71438d86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.857996 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 834/1665] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:30 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 631 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:30.862685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76cceb-4e6a-489b-878f-aec71438d86a admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.866148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76cceb-4e6a-489b-878f-aec71438d86a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.866554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76cceb-4e6a-489b-878f-aec71438d86a admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.868044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76cceb-4e6a-489b-878f-aec71438d86a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:30.870117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76cceb-4e6a-489b-878f-aec71438d86a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:30.871754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68beadb2-e230-45d1-ad35-c1708818a377 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:30.874486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-68beadb2-e230-45d1-ad35-c1708818a377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:30.925512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 832/1666] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:30 2022] POST /identity/v3/users => generated 289 bytes in 784 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:30.927907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76cceb-4e6a-489b-878f-aec71438d86a admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:30.955043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76cceb-4e6a-489b-878f-aec71438d86a admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:30.980841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61358d48-2b7e-48ef-8b19-1c523027d792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.987988 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61358d48-2b7e-48ef-8b19-1c523027d792 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.988429 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61358d48-2b7e-48ef-8b19-1c523027d792 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.988842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61358d48-2b7e-48ef-8b19-1c523027d792 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.989898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61358d48-2b7e-48ef-8b19-1c523027d792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:30.993982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61358d48-2b7e-48ef-8b19-1c523027d792 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:31.037033 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 833/1667] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:31.038219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61358d48-2b7e-48ef-8b19-1c523027d792 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:31.073100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61358d48-2b7e-48ef-8b19-1c523027d792 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:31.117438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a22f76-19cf-45be-aa4f-7781791ef47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:31.118645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a22f76-19cf-45be-aa4f-7781791ef47b admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:31.125774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a22f76-19cf-45be-aa4f-7781791ef47b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:31.126616 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a22f76-19cf-45be-aa4f-7781791ef47b admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:31.128307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a22f76-19cf-45be-aa4f-7781791ef47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:31.138323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a22f76-19cf-45be-aa4f-7781791ef47b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:31.173620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68beadb2-e230-45d1-ad35-c1708818a377 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:31.196779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1e4a7-e0b0-4566-8381-ec3648a235f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:31.197964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1e4a7-e0b0-4566-8381-ec3648a235f0 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:31.201733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1e4a7-e0b0-4566-8381-ec3648a235f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:31.202326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1e4a7-e0b0-4566-8381-ec3648a235f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:31.215234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 834/1668] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:31.221585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a22f76-19cf-45be-aa4f-7781791ef47b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:31.260870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1e4a7-e0b0-4566-8381-ec3648a235f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:31.264879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1e4a7-e0b0-4566-8381-ec3648a235f0 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:31.270779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a22f76-19cf-45be-aa4f-7781791ef47b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:31.295157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39725f6-1a7c-493f-b807-df939492e2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:31.296732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39725f6-1a7c-493f-b807-df939492e2c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:31.297480 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39725f6-1a7c-493f-b807-df939492e2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:31.298926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39725f6-1a7c-493f-b807-df939492e2c2 admin admin] PATH_INFO: `/v3/projects/446294e7931b4032a7b7eb8422a74c38/users/312f2b29f1ca41b7957bafb01326cf9e/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:31.312237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 835/1669] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:30 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:31.344365 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39725f6-1a7c-493f-b807-df939492e2c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=446294e7931b4032a7b7eb8422a74c38, user_id=312f2b29f1ca41b7957bafb01326cf9e, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:31.349503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39725f6-1a7c-493f-b807-df939492e2c2 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:31.446635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 835/1670] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:31 2022] PUT /identity/v3/projects/446294e7931b4032a7b7eb8422a74c38/users/312f2b29f1ca41b7957bafb01326cf9e/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:31.459935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f9241-abee-46d8-96bb-b9895435ce43 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:31.460121 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f9241-abee-46d8-96bb-b9895435ce43 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:31.460403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f9241-abee-46d8-96bb-b9895435ce43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:31.516275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-7a5f9241-abee-46d8-96bb-b9895435ce43 None None] MFA Rules not processed for user `312f2b29f1ca41b7957bafb01326cf9e`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:31.517648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7a5f9241-abee-46d8-96bb-b9895435ce43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:31.720244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 836/1671] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:31.725646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39725f6-1a7c-493f-b807-df939492e2c2 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:31.750547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f9241-abee-46d8-96bb-b9895435ce43 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:31.769142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:31.983899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39725f6-1a7c-493f-b807-df939492e2c2 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:32.001783 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74253bde-1f8b-4902-bb04-2174c5d71fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:32.002831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74253bde-1f8b-4902-bb04-2174c5d71fff admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.003382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74253bde-1f8b-4902-bb04-2174c5d71fff admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.004053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74253bde-1f8b-4902-bb04-2174c5d71fff admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.004188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:32.008646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74253bde-1f8b-4902-bb04-2174c5d71fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:32.011247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74253bde-1f8b-4902-bb04-2174c5d71fff admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:32.023029 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f9241-abee-46d8-96bb-b9895435ce43 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:32.042137 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:32.042833 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.043113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.048630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] PATH_INFO: `/v3/users/4f6699455479413f8ab282d3f5dd3377` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.051947 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f6699455479413f8ab282d3f5dd3377)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:32.065041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:32.119074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 836/1672] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:31 2022] POST /identity/v3/projects => generated 377 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:32.122156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4294c-2af0-46cb-8467-7ce6a11d4617 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.122494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4294c-2af0-46cb-8467-7ce6a11d4617 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.122965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4294c-2af0-46cb-8467-7ce6a11d4617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.226150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b6c4294c-2af0-46cb-8467-7ce6a11d4617 None None] MFA Rules not processed for user `b8818863a81b4238a22035fbb256568e`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:32.235040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f6699455479413f8ab282d3f5dd3377'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:32.237367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b6c4294c-2af0-46cb-8467-7ce6a11d4617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:32.280503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f6699455479413f8ab282d3f5dd3377'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:32.286090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f6699455479413f8ab282d3f5dd3377'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:32.289507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f6699455479413f8ab282d3f5dd3377'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:32.311820 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f6699455479413f8ab282d3f5dd3377'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:32.360562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 837/1673] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:31 2022] DELETE /identity/v3/users/4f6699455479413f8ab282d3f5dd3377 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:32.361553 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:32.365078 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:32.492592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 837/1674] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:32.507558 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896aeecf-cd34-4ed4-bfb9-fede1df060aa None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.507813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896aeecf-cd34-4ed4-bfb9-fede1df060aa None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.508232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896aeecf-cd34-4ed4-bfb9-fede1df060aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.577383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-896aeecf-cd34-4ed4-bfb9-fede1df060aa None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:32.578731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-896aeecf-cd34-4ed4-bfb9-fede1df060aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:32.629699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0f181-f2f4-4a3f-bd52-f53cc3aec293 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:32.653528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d430525-7e7a-4621-b7e4-364bc193f978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:32.654176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d430525-7e7a-4621-b7e4-364bc193f978 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.654533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d430525-7e7a-4621-b7e4-364bc193f978 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.654806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d430525-7e7a-4621-b7e4-364bc193f978 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.655495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d430525-7e7a-4621-b7e4-364bc193f978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:32.663046 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d430525-7e7a-4621-b7e4-364bc193f978 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:32.742401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 838/1675] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:32 2022] POST /identity/v3/users => generated 295 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:32.749919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0719a3-d937-43c7-bc7a-d8dbb83962e8 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.750195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0719a3-d937-43c7-bc7a-d8dbb83962e8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.750475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0719a3-d937-43c7-bc7a-d8dbb83962e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.813590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-cb0719a3-d937-43c7-bc7a-d8dbb83962e8 None None] MFA Rules not processed for user `b8818863a81b4238a22035fbb256568e`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:32.822019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 838/1676] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:32.826474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896aeecf-cd34-4ed4-bfb9-fede1df060aa None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:32.827386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-cb0719a3-d937-43c7-bc7a-d8dbb83962e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:32.856205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896aeecf-cd34-4ed4-bfb9-fede1df060aa None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:32.877820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3719e-5e71-407c-8c3f-4880f212823e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:32.878472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3719e-5e71-407c-8c3f-4880f212823e admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.879430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3719e-5e71-407c-8c3f-4880f212823e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.879430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3719e-5e71-407c-8c3f-4880f212823e admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.880037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3719e-5e71-407c-8c3f-4880f212823e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:32.882426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3719e-5e71-407c-8c3f-4880f212823e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:32.897640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 839/1677] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:32.898666 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3719e-5e71-407c-8c3f-4880f212823e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:32.900500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ebf3719e-5e71-407c-8c3f-4880f212823e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:32.923078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:33.015873 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 839/1678] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:33.018063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0719a3-d937-43c7-bc7a-d8dbb83962e8 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:33.029288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3719e-5e71-407c-8c3f-4880f212823e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:33.042412 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4304b6-bc92-44c9-8c52-a1eea26b85bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:33.045109 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4304b6-bc92-44c9-8c52-a1eea26b85bc service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.045389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4304b6-bc92-44c9-8c52-a1eea26b85bc service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.045657 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4304b6-bc92-44c9-8c52-a1eea26b85bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.069390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0719a3-d937-43c7-bc7a-d8dbb83962e8 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:33.070250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4304b6-bc92-44c9-8c52-a1eea26b85bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:33.072823 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4304b6-bc92-44c9-8c52-a1eea26b85bc service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:33.091848 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a58960-ef22-4ca1-ac8b-316bc9ffe22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:33.095413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a58960-ef22-4ca1-ac8b-316bc9ffe22c admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.096059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a58960-ef22-4ca1-ac8b-316bc9ffe22c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.096781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a58960-ef22-4ca1-ac8b-316bc9ffe22c admin admin] PATH_INFO: `/v3/projects/2369dbc9c167413e99d81cbf889ae607/users/f124f9478fed45feabbe1111bc4f622a/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.101576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 840/1679] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:33.103938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4304b6-bc92-44c9-8c52-a1eea26b85bc service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:33.110021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:33.111873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-aa4304b6-bc92-44c9-8c52-a1eea26b85bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:33.152780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a58960-ef22-4ca1-ac8b-316bc9ffe22c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2369dbc9c167413e99d81cbf889ae607, user_id=f124f9478fed45feabbe1111bc4f622a, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:33.158617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a58960-ef22-4ca1-ac8b-316bc9ffe22c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:33.209630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 840/1680] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:33 2022] PUT /identity/v3/projects/2369dbc9c167413e99d81cbf889ae607/users/f124f9478fed45feabbe1111bc4f622a/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:33.213971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a58960-ef22-4ca1-ac8b-316bc9ffe22c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:33.307810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4304b6-bc92-44c9-8c52-a1eea26b85bc service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:33.530439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a58960-ef22-4ca1-ac8b-316bc9ffe22c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:33.555473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203b57c-552c-4014-8e6a-ca4efa09a6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:33.557314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203b57c-552c-4014-8e6a-ca4efa09a6ec admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.566742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203b57c-552c-4014-8e6a-ca4efa09a6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.567265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203b57c-552c-4014-8e6a-ca4efa09a6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.572897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203b57c-552c-4014-8e6a-ca4efa09a6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:33.585940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203b57c-552c-4014-8e6a-ca4efa09a6ec admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:33.653319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 841/1681] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:33.658294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ca12b-f949-4b67-ba1b-cf37abe4383d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:33.659238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ca12b-f949-4b67-ba1b-cf37abe4383d service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.659690 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ca12b-f949-4b67-ba1b-cf37abe4383d service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.660157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ca12b-f949-4b67-ba1b-cf37abe4383d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.667444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203b57c-552c-4014-8e6a-ca4efa09a6ec admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:33.716481 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203b57c-552c-4014-8e6a-ca4efa09a6ec admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:33.761995 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a151fa7-84fb-42d0-ad42-2703f49ed763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:33.763034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a151fa7-84fb-42d0-ad42-2703f49ed763 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.763457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a151fa7-84fb-42d0-ad42-2703f49ed763 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.763837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a151fa7-84fb-42d0-ad42-2703f49ed763 admin admin] PATH_INFO: `/v3/projects/2369dbc9c167413e99d81cbf889ae607/users/f124f9478fed45feabbe1111bc4f622a/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.771040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a151fa7-84fb-42d0-ad42-2703f49ed763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2369dbc9c167413e99d81cbf889ae607, user_id=f124f9478fed45feabbe1111bc4f622a, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:33.781856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a151fa7-84fb-42d0-ad42-2703f49ed763 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:33.908142 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 842/1682] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:33 2022] PUT /identity/v3/projects/2369dbc9c167413e99d81cbf889ae607/users/f124f9478fed45feabbe1111bc4f622a/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:33.910722 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c1a77-2044-45b9-b66d-8dfad2013aee None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.911190 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c1a77-2044-45b9-b66d-8dfad2013aee None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.919985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c1a77-2044-45b9-b66d-8dfad2013aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.940452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:33.969759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ca12b-f949-4b67-ba1b-cf37abe4383d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:33.982706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ca12b-f949-4b67-ba1b-cf37abe4383d service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:34.016455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 841/1683] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 911 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:34.061506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-c52c1a77-2044-45b9-b66d-8dfad2013aee None None] MFA Rules not processed for user `f124f9478fed45feabbe1111bc4f622a`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:34.062970 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c52c1a77-2044-45b9-b66d-8dfad2013aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:34.350974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 843/1684] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:34.363681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ca12b-f949-4b67-ba1b-cf37abe4383d service nova] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:34.365780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-580ca12b-f949-4b67-ba1b-cf37abe4383d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:34.533122 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ca12b-f949-4b67-ba1b-cf37abe4383d service nova] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:34.669147 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe3679-b2e7-4b41-8882-9e03d6828f95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:34.669694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe3679-b2e7-4b41-8882-9e03d6828f95 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:34.669971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe3679-b2e7-4b41-8882-9e03d6828f95 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:34.670227 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe3679-b2e7-4b41-8882-9e03d6828f95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:34.684415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-19fe3679-b2e7-4b41-8882-9e03d6828f95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:34.813824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe3679-b2e7-4b41-8882-9e03d6828f95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:34.815338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe3679-b2e7-4b41-8882-9e03d6828f95 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:34.834755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 842/1685] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:35.001565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b5c16-d42e-4bca-9310-113221081ad8 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:35.002420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b5c16-d42e-4bca-9310-113221081ad8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:35.002880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b5c16-d42e-4bca-9310-113221081ad8 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:35.005113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 844/1686] 10.222.0.36 () {54 vars in 912 bytes} [Wed Aug 31 09:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:35.020115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a87a5d-973d-413f-b397-52a95c93106e None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:35.020592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a87a5d-973d-413f-b397-52a95c93106e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:35.021075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a87a5d-973d-413f-b397-52a95c93106e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:35.176067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f2a87a5d-973d-413f-b397-52a95c93106e None None] MFA Rules not processed for user `6ff1177119cb48e0a88b0f11a6b1b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:35.187879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f2a87a5d-973d-413f-b397-52a95c93106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:35.319522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:35.343744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 843/1687] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:48:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:35.440630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b5c16-d42e-4bca-9310-113221081ad8 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:35.463908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b5c16-d42e-4bca-9310-113221081ad8 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:35.476108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab9af6-f830-451b-83bc-58431597df2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:35.476840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab9af6-f830-451b-83bc-58431597df2e service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:35.477185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab9af6-f830-451b-83bc-58431597df2e service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:35.477525 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab9af6-f830-451b-83bc-58431597df2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:35.498055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab9af6-f830-451b-83bc-58431597df2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:35.499935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab9af6-f830-451b-83bc-58431597df2e service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:35.552239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 845/1688] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:35.945054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a87a5d-973d-413f-b397-52a95c93106e None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:35.948197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f2a87a5d-973d-413f-b397-52a95c93106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:35.958420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab9af6-f830-451b-83bc-58431597df2e service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:35.961680 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1cab9af6-f830-451b-83bc-58431597df2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:36.180080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:36.243968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab9af6-f830-451b-83bc-58431597df2e service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:36.257924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:36.259292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:36.259766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:36.260254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] PATH_INFO: `/v3/projects/e6653107d0a44ddfb81af0035d6b0640` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:36.263366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6653107d0a44ddfb81af0035d6b0640)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:36.283971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:36.339293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a87a5d-973d-413f-b397-52a95c93106e None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:36.356581 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ce5ef-c4bd-49a2-a8be-c8f24f18acdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:36.359194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ce5ef-c4bd-49a2-a8be-c8f24f18acdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:36.359760 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ce5ef-c4bd-49a2-a8be-c8f24f18acdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:36.360154 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ce5ef-c4bd-49a2-a8be-c8f24f18acdb admin admin] PATH_INFO: `/v3/users/9ba257f98cb64603a82b1c702659bbcd` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:36.367833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ce5ef-c4bd-49a2-a8be-c8f24f18acdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ba257f98cb64603a82b1c702659bbcd)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:36.372790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ce5ef-c4bd-49a2-a8be-c8f24f18acdb admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:36.398031 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6653107d0a44ddfb81af0035d6b0640', 'request_id': 'req-d587d21f-8017-4d97-9eac-6aff3175dd31'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.408703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6653107d0a44ddfb81af0035d6b0640', 'request_id': 'req-d587d21f-8017-4d97-9eac-6aff3175dd31'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.441586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6653107d0a44ddfb81af0035d6b0640', 'request_id': 'req-d587d21f-8017-4d97-9eac-6aff3175dd31'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.466576 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] The token cache is being invalidate because project e6653107d0a44ddfb81af0035d6b0640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:36.469993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.472358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.478376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 846/1689] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:35 2022] DELETE /identity/v3/projects/e6653107d0a44ddfb81af0035d6b0640 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:36.488759 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:36.492447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:36.576132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-bf2ce5ef-c4bd-49a2-a8be-c8f24f18acdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ba257f98cb64603a82b1c702659bbcd'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.611698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-bf2ce5ef-c4bd-49a2-a8be-c8f24f18acdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ba257f98cb64603a82b1c702659bbcd'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.613680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-bf2ce5ef-c4bd-49a2-a8be-c8f24f18acdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ba257f98cb64603a82b1c702659bbcd'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.616209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587d21f-8017-4d97-9eac-6aff3175dd31 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:36.631386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:36.632058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:36.632346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:36.632628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] PATH_INFO: `/v3/users/5630dc664041438c95af9b555e4e91f9` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:36.638004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] RBAC: Authorizing `identity:delete_user(user_id=5630dc664041438c95af9b555e4e91f9)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:36.640512 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:36.665809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-bf2ce5ef-c4bd-49a2-a8be-c8f24f18acdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ba257f98cb64603a82b1c702659bbcd'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.669168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-bf2ce5ef-c4bd-49a2-a8be-c8f24f18acdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ba257f98cb64603a82b1c702659bbcd'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.714087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 844/1690] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:35 2022] DELETE /identity/v3/users/9ba257f98cb64603a82b1c702659bbcd => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:36.721363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5e519-4d7c-4931-8498-c53bd3ec25fb None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:36.721363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5e519-4d7c-4931-8498-c53bd3ec25fb None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:36.721363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5e519-4d7c-4931-8498-c53bd3ec25fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:36.765451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5630dc664041438c95af9b555e4e91f9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.794002 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5630dc664041438c95af9b555e4e91f9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.794696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5630dc664041438c95af9b555e4e91f9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.796316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-58e5e519-4d7c-4931-8498-c53bd3ec25fb None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:36.801134 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-58e5e519-4d7c-4931-8498-c53bd3ec25fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:36.804003 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5630dc664041438c95af9b555e4e91f9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.816454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5630dc664041438c95af9b555e4e91f9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:36.819431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:36.852206 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 847/1691] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:36 2022] DELETE /identity/v3/users/5630dc664041438c95af9b555e4e91f9 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:37.022222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 845/1692] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:37.151615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:37.154166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:37.369044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b2c8c-f68d-48a3-8a61-eb846a04adce admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:37.376596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115d864-4789-4013-8e30-dd768ce99edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:37.376840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115d864-4789-4013-8e30-dd768ce99edc service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:37.377098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115d864-4789-4013-8e30-dd768ce99edc service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:37.377464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115d864-4789-4013-8e30-dd768ce99edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:37.545680 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115d864-4789-4013-8e30-dd768ce99edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:37.551371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115d864-4789-4013-8e30-dd768ce99edc service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:37.587944 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 848/1693] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:37.637312 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560d33c-65a9-48d8-937c-c0821053a010 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:37.637599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560d33c-65a9-48d8-937c-c0821053a010 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:37.637880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560d33c-65a9-48d8-937c-c0821053a010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:37.710003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-3560d33c-65a9-48d8-937c-c0821053a010 None None] MFA Rules not processed for user `8a3df25e740c4c7f8a311d5a1574c3ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:37.712461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3560d33c-65a9-48d8-937c-c0821053a010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:37.776775 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:37.930686 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c95e5-ee41-4a72-be8a-24a76b817b29 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:37.931395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c95e5-ee41-4a72-be8a-24a76b817b29 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:37.931819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c95e5-ee41-4a72-be8a-24a76b817b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:37.957748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 846/1694] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:37.976821 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ae67c-f889-4ecb-a172-cccb43f63126 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:37.977406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ae67c-f889-4ecb-a172-cccb43f63126 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:37.977820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ae67c-f889-4ecb-a172-cccb43f63126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:38.001445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-1f4ae67c-f889-4ecb-a172-cccb43f63126 None None] MFA Rules not processed for user `8a3df25e740c4c7f8a311d5a1574c3ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:38.003040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1f4ae67c-f889-4ecb-a172-cccb43f63126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:38.017411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 847/1695] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:38.031382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ae67c-f889-4ecb-a172-cccb43f63126 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:38.034090 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1f4ae67c-f889-4ecb-a172-cccb43f63126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:38.078758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-797c95e5-ee41-4a72-be8a-24a76b817b29 None None] MFA Rules not processed for user `b6ca4276e1554d4184451313b2e09a88`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:38.080234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-797c95e5-ee41-4a72-be8a-24a76b817b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:38.195269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ae67c-f889-4ecb-a172-cccb43f63126 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:38.203103 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6967472-129f-4468-ab6f-830d7b222890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:38.210484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6967472-129f-4468-ab6f-830d7b222890 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:38.211224 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6967472-129f-4468-ab6f-830d7b222890 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:38.211655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6967472-129f-4468-ab6f-830d7b222890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:38.224842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6967472-129f-4468-ab6f-830d7b222890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:38.230985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6967472-129f-4468-ab6f-830d7b222890 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:38.268202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 848/1696] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:38.270635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6967472-129f-4468-ab6f-830d7b222890 service nova] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:38.275249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b6967472-129f-4468-ab6f-830d7b222890 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:38.282471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 849/1697] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:38.299337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d435b-de6b-4eca-817e-ac2554a9cf40 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:38.299903 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d435b-de6b-4eca-817e-ac2554a9cf40 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:38.300327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d435b-de6b-4eca-817e-ac2554a9cf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:38.352978 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-288d435b-de6b-4eca-817e-ac2554a9cf40 None None] MFA Rules not processed for user `b8818863a81b4238a22035fbb256568e`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:38.354797 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-288d435b-de6b-4eca-817e-ac2554a9cf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:38.397430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6967472-129f-4468-ab6f-830d7b222890 service nova] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:38.404723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f69421-e359-43e4-9cb4-f624bef05dca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:38.405367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f69421-e359-43e4-9cb4-f624bef05dca service placement] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:38.405660 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f69421-e359-43e4-9cb4-f624bef05dca service placement] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:38.405932 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f69421-e359-43e4-9cb4-f624bef05dca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:38.407978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c6f69421-e359-43e4-9cb4-f624bef05dca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:38.424273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f69421-e359-43e4-9cb4-f624bef05dca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:38.426133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f69421-e359-43e4-9cb4-f624bef05dca service placement] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:38.450411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 849/1698] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:38.453444 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f69421-e359-43e4-9cb4-f624bef05dca service placement] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:38.458228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c6f69421-e359-43e4-9cb4-f624bef05dca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:38.493475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:38.507222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f69421-e359-43e4-9cb4-f624bef05dca service placement] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:38.523454 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53710244-c162-4eed-8b42-8f942f2d8d33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:38.524807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53710244-c162-4eed-8b42-8f942f2d8d33 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:38.526735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53710244-c162-4eed-8b42-8f942f2d8d33 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:38.528805 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53710244-c162-4eed-8b42-8f942f2d8d33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:38.558096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53710244-c162-4eed-8b42-8f942f2d8d33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:38.569284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53710244-c162-4eed-8b42-8f942f2d8d33 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:38.585473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 850/1699] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:38.600155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 850/1700] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:38.973891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d435b-de6b-4eca-817e-ac2554a9cf40 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:38.976173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-288d435b-de6b-4eca-817e-ac2554a9cf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:39.053847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d435b-de6b-4eca-817e-ac2554a9cf40 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:39.077360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfe032-611f-4ae3-903a-4a2f4c29a460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:39.083552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfe032-611f-4ae3-903a-4a2f4c29a460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:39.084358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfe032-611f-4ae3-903a-4a2f4c29a460 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:39.088863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfe032-611f-4ae3-903a-4a2f4c29a460 admin admin] PATH_INFO: `/v3/projects/76cf84eb16374a3d92db67eb3186d394` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:39.094294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfe032-611f-4ae3-903a-4a2f4c29a460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76cf84eb16374a3d92db67eb3186d394)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:39.121420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfe032-611f-4ae3-903a-4a2f4c29a460 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:39.248846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8f0c6-09e5-4fda-a351-1fb90e1cb980 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:39.249422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8f0c6-09e5-4fda-a351-1fb90e1cb980 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:39.249790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8f0c6-09e5-4fda-a351-1fb90e1cb980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:39.290683 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:39.360825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-58c8f0c6-09e5-4fda-a351-1fb90e1cb980 None None] MFA Rules not processed for user `980ab18eb5c646378d738079621a0dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:39.362928 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-58c8f0c6-09e5-4fda-a351-1fb90e1cb980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:39.381767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-95dfe032-611f-4ae3-903a-4a2f4c29a460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76cf84eb16374a3d92db67eb3186d394', 'request_id': 'req-95dfe032-611f-4ae3-903a-4a2f4c29a460'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:39.405185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-95dfe032-611f-4ae3-903a-4a2f4c29a460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76cf84eb16374a3d92db67eb3186d394', 'request_id': 'req-95dfe032-611f-4ae3-903a-4a2f4c29a460'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:39.420043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-95dfe032-611f-4ae3-903a-4a2f4c29a460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76cf84eb16374a3d92db67eb3186d394', 'request_id': 'req-95dfe032-611f-4ae3-903a-4a2f4c29a460'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:39.452773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-95dfe032-611f-4ae3-903a-4a2f4c29a460 admin admin] The token cache is being invalidate because project 76cf84eb16374a3d92db67eb3186d394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:39.453139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-95dfe032-611f-4ae3-903a-4a2f4c29a460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:39.457030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-95dfe032-611f-4ae3-903a-4a2f4c29a460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:39.465558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 851/1701] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:38 2022] DELETE /identity/v3/projects/76cf84eb16374a3d92db67eb3186d394 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:39.470565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d81c6-f74a-45b2-8485-a02e075021c8 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:39.472521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d81c6-f74a-45b2-8485-a02e075021c8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:39.472882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d81c6-f74a-45b2-8485-a02e075021c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:39.543621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-2b7d81c6-f74a-45b2-8485-a02e075021c8 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:39.551623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2b7d81c6-f74a-45b2-8485-a02e075021c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:39.582713 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 851/1702] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:39.688801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 852/1703] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:39.695512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da645c-d298-4997-b657-a65a93efcbc4 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:39.696691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da645c-d298-4997-b657-a65a93efcbc4 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:39.696993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da645c-d298-4997-b657-a65a93efcbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:39.739100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-23da645c-d298-4997-b657-a65a93efcbc4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:39.741503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-23da645c-d298-4997-b657-a65a93efcbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:39.775751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 852/1704] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:39.780194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d81c6-f74a-45b2-8485-a02e075021c8 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:39.817035 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d81c6-f74a-45b2-8485-a02e075021c8 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:39.830033 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abadae-e78f-4d02-b592-5f402a0e6e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:39.830859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abadae-e78f-4d02-b592-5f402a0e6e80 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:39.831384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abadae-e78f-4d02-b592-5f402a0e6e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:39.831751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abadae-e78f-4d02-b592-5f402a0e6e80 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:39.832541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7abadae-e78f-4d02-b592-5f402a0e6e80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:39.832948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7abadae-e78f-4d02-b592-5f402a0e6e80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:39.836544 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7abadae-e78f-4d02-b592-5f402a0e6e80 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:39.888636 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da645c-d298-4997-b657-a65a93efcbc4 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:39.889338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 853/1705] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:39.904938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da645c-d298-4997-b657-a65a93efcbc4 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:39.915265 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4159b-36fe-4a85-9e4d-1773a8f4c85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:39.916346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4159b-36fe-4a85-9e4d-1773a8f4c85c admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:39.916859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4159b-36fe-4a85-9e4d-1773a8f4c85c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:39.917275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4159b-36fe-4a85-9e4d-1773a8f4c85c admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:39.918433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4159b-36fe-4a85-9e4d-1773a8f4c85c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:39.921859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4159b-36fe-4a85-9e4d-1773a8f4c85c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:39.993688 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 853/1706] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:39 2022] POST /identity/v3/projects => generated 393 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:39.999715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abadae-e78f-4d02-b592-5f402a0e6e80 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:40.042123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abadae-e78f-4d02-b592-5f402a0e6e80 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:40.065169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4159b-36fe-4a85-9e4d-1773a8f4c85c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:40.067224 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-68f4159b-36fe-4a85-9e4d-1773a8f4c85c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:40.202021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:40.232057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4961213-8f7e-42f8-8d1b-fb0fcf7523a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:40.232876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4961213-8f7e-42f8-8d1b-fb0fcf7523a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:40.233336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4961213-8f7e-42f8-8d1b-fb0fcf7523a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:40.233758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4961213-8f7e-42f8-8d1b-fb0fcf7523a1 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:40.234562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4961213-8f7e-42f8-8d1b-fb0fcf7523a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:40.237232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4961213-8f7e-42f8-8d1b-fb0fcf7523a1 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:40.271010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 854/1707] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:39 2022] POST /identity/v3/users => generated 297 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:40.278100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4961213-8f7e-42f8-8d1b-fb0fcf7523a1 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:40.302800 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4961213-8f7e-42f8-8d1b-fb0fcf7523a1 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:40.308777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d09e6-decd-4306-b39b-1059b182a6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:40.309596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d09e6-decd-4306-b39b-1059b182a6ab admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:40.309880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d09e6-decd-4306-b39b-1059b182a6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:40.310159 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d09e6-decd-4306-b39b-1059b182a6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:40.310834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d09e6-decd-4306-b39b-1059b182a6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:40.313511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d09e6-decd-4306-b39b-1059b182a6ab admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:40.348619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4159b-36fe-4a85-9e4d-1773a8f4c85c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:40.351988 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 855/1708] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:40.353354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d09e6-decd-4306-b39b-1059b182a6ab admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:40.365502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:40.368025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:40.368519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:40.368945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] PATH_INFO: `/v3/projects/e5b56cc9afe641ae8f21b59f77301f1e` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:40.370479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5b56cc9afe641ae8f21b59f77301f1e)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:40.373062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:40.376424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-863d09e6-decd-4306-b39b-1059b182a6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:40.479981 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5b56cc9afe641ae8f21b59f77301f1e', 'request_id': 'req-dd049645-eb7a-409e-b228-657a7ce68e4e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.490428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5b56cc9afe641ae8f21b59f77301f1e', 'request_id': 'req-dd049645-eb7a-409e-b228-657a7ce68e4e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.504901 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5b56cc9afe641ae8f21b59f77301f1e', 'request_id': 'req-dd049645-eb7a-409e-b228-657a7ce68e4e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.514449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:40.536266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] The token cache is being invalidate because project e5b56cc9afe641ae8f21b59f77301f1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:40.536699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.545507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.548610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 854/1709] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:40 2022] DELETE /identity/v3/projects/e5b56cc9afe641ae8f21b59f77301f1e => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:40.556770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:40.563077 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:40.783734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd049645-eb7a-409e-b228-657a7ce68e4e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:40.791820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ac49e-4b47-42b7-a89a-87e4b05bda56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:40.792511 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ac49e-4b47-42b7-a89a-87e4b05bda56 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:40.792717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ac49e-4b47-42b7-a89a-87e4b05bda56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:40.793065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ac49e-4b47-42b7-a89a-87e4b05bda56 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:40.793755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ac49e-4b47-42b7-a89a-87e4b05bda56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:40.795477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ac49e-4b47-42b7-a89a-87e4b05bda56 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:40.814755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d09e6-decd-4306-b39b-1059b182a6ab admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:40.820613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 855/1710] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:40.821989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bdc39-4100-4f2f-bdff-37635e76a797 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:40.825428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bdc39-4100-4f2f-bdff-37635e76a797 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:40.825639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bdc39-4100-4f2f-bdff-37635e76a797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:40.892751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-550bdc39-4100-4f2f-bdff-37635e76a797 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:40.894328 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-550bdc39-4100-4f2f-bdff-37635e76a797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:40.916539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 856/1711] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:40.917417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550bdc39-4100-4f2f-bdff-37635e76a797 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:40.923678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-550bdc39-4100-4f2f-bdff-37635e76a797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:41.061653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550bdc39-4100-4f2f-bdff-37635e76a797 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:41.065196 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1b9fe-041b-459d-a79b-87cd78ce9d40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:41.065707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f15590-cd48-4769-9d85-793e7f58228d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:41.066468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f15590-cd48-4769-9d85-793e7f58228d service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:41.066747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f15590-cd48-4769-9d85-793e7f58228d service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:41.067029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f15590-cd48-4769-9d85-793e7f58228d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:41.068057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1b9fe-041b-459d-a79b-87cd78ce9d40 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:41.073316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-25f15590-cd48-4769-9d85-793e7f58228d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:41.084711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1b9fe-041b-459d-a79b-87cd78ce9d40 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:41.086625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1b9fe-041b-459d-a79b-87cd78ce9d40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:41.095192 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-09b1b9fe-041b-459d-a79b-87cd78ce9d40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:41.320667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f15590-cd48-4769-9d85-793e7f58228d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:41.326188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f15590-cd48-4769-9d85-793e7f58228d service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:41.359535 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1b9fe-041b-459d-a79b-87cd78ce9d40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:41.364501 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1b9fe-041b-459d-a79b-87cd78ce9d40 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:41.384664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:41.391493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 857/1712] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:41.395741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f15590-cd48-4769-9d85-793e7f58228d service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:41.423598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 856/1713] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 1071 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:41.425140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a6a78-0167-4c6a-9a79-4d5c20924b8a None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:41.425425 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a6a78-0167-4c6a-9a79-4d5c20924b8a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:41.425699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a6a78-0167-4c6a-9a79-4d5c20924b8a None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:41.430433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 857/1714] 10.222.0.36 () {54 vars in 912 bytes} [Wed Aug 31 09:48:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:41.431764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6593a-555e-4d96-adcc-edb2e108f0e7 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:41.432048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6593a-555e-4d96-adcc-edb2e108f0e7 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:41.432320 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6593a-555e-4d96-adcc-edb2e108f0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:41.436042 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f15590-cd48-4769-9d85-793e7f58228d service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:41.459452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e157e-542a-490c-bbd7-de790c4c1ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:41.459452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e157e-542a-490c-bbd7-de790c4c1ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:41.459452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e157e-542a-490c-bbd7-de790c4c1ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:41.464947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e157e-542a-490c-bbd7-de790c4c1ae2 admin admin] PATH_INFO: `/v3/projects/5131f516ee1c473fbac6c74b1d73ddcc/users/47aecbc7a7e146d6ad996d69de0d7beb/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:41.525281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e157e-542a-490c-bbd7-de790c4c1ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5131f516ee1c473fbac6c74b1d73ddcc, user_id=47aecbc7a7e146d6ad996d69de0d7beb, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:41.528150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e157e-542a-490c-bbd7-de790c4c1ae2 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:41.547483 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-31b6593a-555e-4d96-adcc-edb2e108f0e7 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:41.549736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-31b6593a-555e-4d96-adcc-edb2e108f0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:41.590005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 858/1715] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:41.592297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e2b13-f565-4e8d-b5ac-05cd04b6e70d None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:41.592885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e2b13-f565-4e8d-b5ac-05cd04b6e70d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:41.593718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e2b13-f565-4e8d-b5ac-05cd04b6e70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:41.597825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 858/1716] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:41 2022] PUT /identity/v3/projects/5131f516ee1c473fbac6c74b1d73ddcc/users/47aecbc7a7e146d6ad996d69de0d7beb/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:41.603274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e157e-542a-490c-bbd7-de790c4c1ae2 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:41.658951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:41.676964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-a04e2b13-f565-4e8d-b5ac-05cd04b6e70d None None] MFA Rules not processed for user `6ff1177119cb48e0a88b0f11a6b1b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:41.680582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a04e2b13-f565-4e8d-b5ac-05cd04b6e70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:41.871998 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e157e-542a-490c-bbd7-de790c4c1ae2 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:41.887519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7cc45-19c2-40cc-8035-719758194244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:41.891978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7cc45-19c2-40cc-8035-719758194244 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:41.891978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7cc45-19c2-40cc-8035-719758194244 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:41.891978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7cc45-19c2-40cc-8035-719758194244 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:41.898368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7cc45-19c2-40cc-8035-719758194244 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:41.898368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7cc45-19c2-40cc-8035-719758194244 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:41.909273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7cc45-19c2-40cc-8035-719758194244 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:42.027784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 859/1717] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:48:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:42.028616 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 859/1718] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:42.030107 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a98413-0619-482e-9ba9-aae060e87fc4 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.030657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a98413-0619-482e-9ba9-aae060e87fc4 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.031016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a98413-0619-482e-9ba9-aae060e87fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.032312 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7cc45-19c2-40cc-8035-719758194244 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:42.086106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7cc45-19c2-40cc-8035-719758194244 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:42.133749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c6b29-c25a-4194-8ab6-46173c16774c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:42.134818 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c6b29-c25a-4194-8ab6-46173c16774c admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.137219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c6b29-c25a-4194-8ab6-46173c16774c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.142881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c6b29-c25a-4194-8ab6-46173c16774c admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.148518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c6b29-c25a-4194-8ab6-46173c16774c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:42.161630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c6b29-c25a-4194-8ab6-46173c16774c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:42.200311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-94a98413-0619-482e-9ba9-aae060e87fc4 None None] MFA Rules not processed for user `47aecbc7a7e146d6ad996d69de0d7beb`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:42.213144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-94a98413-0619-482e-9ba9-aae060e87fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:42.337698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 860/1719] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:42 2022] POST /identity/v3/projects => generated 403 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:42.346167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c6b29-c25a-4194-8ab6-46173c16774c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:42.360316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-814c6b29-c25a-4194-8ab6-46173c16774c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:42.595787 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c6b29-c25a-4194-8ab6-46173c16774c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:42.598867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 860/1720] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 570 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:42.601234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a98413-0619-482e-9ba9-aae060e87fc4 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:42.610341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b6f13-0a0d-44a9-bb37-9d98b9bf7ab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:42.610990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b6f13-0a0d-44a9-bb37-9d98b9bf7ab9 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.611300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b6f13-0a0d-44a9-bb37-9d98b9bf7ab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.611576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b6f13-0a0d-44a9-bb37-9d98b9bf7ab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.638231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b6f13-0a0d-44a9-bb37-9d98b9bf7ab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:42.640347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b6f13-0a0d-44a9-bb37-9d98b9bf7ab9 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:42.654075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 861/1721] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:42.655270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b6f13-0a0d-44a9-bb37-9d98b9bf7ab9 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:42.682837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a98413-0619-482e-9ba9-aae060e87fc4 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:42.945640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b6f13-0a0d-44a9-bb37-9d98b9bf7ab9 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:42.953442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295f12e-ccd1-40b1-acde-e77527e08676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:42.954095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295f12e-ccd1-40b1-acde-e77527e08676 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.960172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295f12e-ccd1-40b1-acde-e77527e08676 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.961528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295f12e-ccd1-40b1-acde-e77527e08676 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.962940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295f12e-ccd1-40b1-acde-e77527e08676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:42.970059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295f12e-ccd1-40b1-acde-e77527e08676 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.013568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a15c9-754f-4ce7-8d92-1435002f58e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.017686 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a15c9-754f-4ce7-8d92-1435002f58e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.019058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a15c9-754f-4ce7-8d92-1435002f58e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.024523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a15c9-754f-4ce7-8d92-1435002f58e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.029564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a15c9-754f-4ce7-8d92-1435002f58e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.036207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a15c9-754f-4ce7-8d92-1435002f58e5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.066249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 862/1722] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:42 2022] POST /identity/v3/projects => generated 391 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:43.073227 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295f12e-ccd1-40b1-acde-e77527e08676 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.132477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 861/1723] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:42 2022] POST /identity/v3/users => generated 302 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:43.138613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a15c9-754f-4ce7-8d92-1435002f58e5 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.162105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:43.164772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a15c9-754f-4ce7-8d92-1435002f58e5 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.218213 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:43.519532 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f34af6-53e7-448b-b112-bc438ad2d2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.519847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f34af6-53e7-448b-b112-bc438ad2d2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.520038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295f12e-ccd1-40b1-acde-e77527e08676 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.520222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f34af6-53e7-448b-b112-bc438ad2d2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.520396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f34af6-53e7-448b-b112-bc438ad2d2b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.521132 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f34af6-53e7-448b-b112-bc438ad2d2b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.529131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f34af6-53e7-448b-b112-bc438ad2d2b9 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.530816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e3d54-eef7-43a4-8d6b-d9d6dc9c0040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.532909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e3d54-eef7-43a4-8d6b-d9d6dc9c0040 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.533545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e3d54-eef7-43a4-8d6b-d9d6dc9c0040 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.534145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e3d54-eef7-43a4-8d6b-d9d6dc9c0040 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.538362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e3d54-eef7-43a4-8d6b-d9d6dc9c0040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.544274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e3d54-eef7-43a4-8d6b-d9d6dc9c0040 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.599052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 862/1724] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:43.607141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f34af6-53e7-448b-b112-bc438ad2d2b9 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.637510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f34af6-53e7-448b-b112-bc438ad2d2b9 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.656396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b68fee-3505-4a01-9def-7b8525e0d213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.657034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b68fee-3505-4a01-9def-7b8525e0d213 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.657318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b68fee-3505-4a01-9def-7b8525e0d213 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.657748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b68fee-3505-4a01-9def-7b8525e0d213 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.658555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b68fee-3505-4a01-9def-7b8525e0d213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.662702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 863/1725] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:43 2022] POST /identity/v3/users => generated 302 bytes in 591 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:43.667097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e3d54-eef7-43a4-8d6b-d9d6dc9c0040 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.672674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b68fee-3505-4a01-9def-7b8525e0d213 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.707837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 863/1726] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:43.715333 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b68fee-3505-4a01-9def-7b8525e0d213 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.737744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b68fee-3505-4a01-9def-7b8525e0d213 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.738352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e3d54-eef7-43a4-8d6b-d9d6dc9c0040 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.751261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cf697-c549-4ac6-8766-c0f2d6ef91ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.752019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cf697-c549-4ac6-8766-c0f2d6ef91ea admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.752369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cf697-c549-4ac6-8766-c0f2d6ef91ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.754202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cf697-c549-4ac6-8766-c0f2d6ef91ea admin admin] PATH_INFO: `/v3/projects/742f1c1cfb7c4116b40bacb83e7a43dd/users/77b5de46807b4c2ba9eb6a874da14b2b/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.770222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957a809-af19-447f-937e-b6adc40d9198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.782965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957a809-af19-447f-937e-b6adc40d9198 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.783591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957a809-af19-447f-937e-b6adc40d9198 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.784086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957a809-af19-447f-937e-b6adc40d9198 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.785555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957a809-af19-447f-937e-b6adc40d9198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.788840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957a809-af19-447f-937e-b6adc40d9198 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.803937 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0cf697-c549-4ac6-8766-c0f2d6ef91ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=742f1c1cfb7c4116b40bacb83e7a43dd, user_id=77b5de46807b4c2ba9eb6a874da14b2b, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.817246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0cf697-c549-4ac6-8766-c0f2d6ef91ea admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.864001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 864/1727] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:48:43.865785 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 864/1728] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:43 2022] PUT /identity/v3/projects/742f1c1cfb7c4116b40bacb83e7a43dd/users/77b5de46807b4c2ba9eb6a874da14b2b/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:43.872617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957a809-af19-447f-937e-b6adc40d9198 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.873206 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fb8b9-4df8-4681-ac43-fa2612719b95 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.873517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fb8b9-4df8-4681-ac43-fa2612719b95 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.874868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fb8b9-4df8-4681-ac43-fa2612719b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.912679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957a809-af19-447f-937e-b6adc40d9198 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.999549 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-504fb8b9-4df8-4681-ac43-fa2612719b95 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:44.001086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-504fb8b9-4df8-4681-ac43-fa2612719b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:44.203352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 865/1729] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:44.228439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d205ad-bbe0-4997-9dc8-2667176b5360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:44.229565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d205ad-bbe0-4997-9dc8-2667176b5360 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:44.230057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d205ad-bbe0-4997-9dc8-2667176b5360 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:44.230548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d205ad-bbe0-4997-9dc8-2667176b5360 admin admin] PATH_INFO: `/v3/projects/c50ca06f92084817ba5c30386532c15d/users/ead51d4314f446ccbe0b4d626d424a64/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.260447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fb8b9-4df8-4681-ac43-fa2612719b95 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:44.334689 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d205ad-bbe0-4997-9dc8-2667176b5360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c50ca06f92084817ba5c30386532c15d, user_id=ead51d4314f446ccbe0b4d626d424a64, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:44.348309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d205ad-bbe0-4997-9dc8-2667176b5360 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:44.409582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fb8b9-4df8-4681-ac43-fa2612719b95 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:44.426064 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a3cbb-986a-4b5e-b99f-8ff1c90b2172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:44.426859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a3cbb-986a-4b5e-b99f-8ff1c90b2172 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:44.427220 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a3cbb-986a-4b5e-b99f-8ff1c90b2172 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:44.427501 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a3cbb-986a-4b5e-b99f-8ff1c90b2172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.445320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 865/1730] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:43 2022] PUT /identity/v3/projects/c50ca06f92084817ba5c30386532c15d/users/ead51d4314f446ccbe0b4d626d424a64/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:44.451820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d205ad-bbe0-4997-9dc8-2667176b5360 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:44.459665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a3cbb-986a-4b5e-b99f-8ff1c90b2172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:44.465585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a3cbb-986a-4b5e-b99f-8ff1c90b2172 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:44.505710 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 866/1731] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:44.737431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d205ad-bbe0-4997-9dc8-2667176b5360 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:44.778850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf8ff6-7fee-47a8-ae73-ca522e85d3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:44.782676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf8ff6-7fee-47a8-ae73-ca522e85d3b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:44.782969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf8ff6-7fee-47a8-ae73-ca522e85d3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:44.783290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf8ff6-7fee-47a8-ae73-ca522e85d3b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.783994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf8ff6-7fee-47a8-ae73-ca522e85d3b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:44.793374 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf8ff6-7fee-47a8-ae73-ca522e85d3b6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:44.864205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a3cbb-986a-4b5e-b99f-8ff1c90b2172 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:44.867617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 866/1732] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:44.886480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a3cbb-986a-4b5e-b99f-8ff1c90b2172 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:44.914762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c30db-05b2-48d6-bd14-a421c15dccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:44.914762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c30db-05b2-48d6-bd14-a421c15dccb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:44.914762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c30db-05b2-48d6-bd14-a421c15dccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:44.914762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c30db-05b2-48d6-bd14-a421c15dccb4 admin admin] PATH_INFO: `/v3/projects/c50ca06f92084817ba5c30386532c15d/users/ead51d4314f446ccbe0b4d626d424a64/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.923968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c30db-05b2-48d6-bd14-a421c15dccb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c50ca06f92084817ba5c30386532c15d, user_id=ead51d4314f446ccbe0b4d626d424a64, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:44.938119 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c30db-05b2-48d6-bd14-a421c15dccb4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:45.032268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 867/1733] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:44 2022] PUT /identity/v3/projects/c50ca06f92084817ba5c30386532c15d/users/ead51d4314f446ccbe0b4d626d424a64/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:45.034105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15778614-773f-49f1-b617-96b4c6dbed6f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:45.034597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15778614-773f-49f1-b617-96b4c6dbed6f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:45.035053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15778614-773f-49f1-b617-96b4c6dbed6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:45.097083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:45.165436 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-15778614-773f-49f1-b617-96b4c6dbed6f None None] MFA Rules not processed for user `ead51d4314f446ccbe0b4d626d424a64`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:45.170782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-15778614-773f-49f1-b617-96b4c6dbed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:45.436776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c30db-05b2-48d6-bd14-a421c15dccb4 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:45.438239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 867/1734] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:45 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:45.461888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:45.657862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c30db-05b2-48d6-bd14-a421c15dccb4 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:45.668105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b8a7e-66ee-4a8d-8e70-e2685fd4c5d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:45.669493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b8a7e-66ee-4a8d-8e70-e2685fd4c5d2 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:45.669666 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b8a7e-66ee-4a8d-8e70-e2685fd4c5d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:45.670109 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b8a7e-66ee-4a8d-8e70-e2685fd4c5d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:45.721212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b8a7e-66ee-4a8d-8e70-e2685fd4c5d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:45.723413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b8a7e-66ee-4a8d-8e70-e2685fd4c5d2 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:45.815419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 868/1735] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:45 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:46.023222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15778614-773f-49f1-b617-96b4c6dbed6f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:46.025662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-15778614-773f-49f1-b617-96b4c6dbed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:46.085049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15778614-773f-49f1-b617-96b4c6dbed6f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:46.105768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9704a4-2237-465e-b4fd-46b91023136f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:46.107221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9704a4-2237-465e-b4fd-46b91023136f service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.107934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9704a4-2237-465e-b4fd-46b91023136f service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.108593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9704a4-2237-465e-b4fd-46b91023136f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.446960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9704a4-2237-465e-b4fd-46b91023136f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:46.449307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9704a4-2237-465e-b4fd-46b91023136f service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:46.480140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 868/1736] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:46 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:46.760718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba15e4-30b9-4e87-a1d0-993056e32f01 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.761546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba15e4-30b9-4e87-a1d0-993056e32f01 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.761546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba15e4-30b9-4e87-a1d0-993056e32f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.029310 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-52ba15e4-30b9-4e87-a1d0-993056e32f01 None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:47.039603 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-52ba15e4-30b9-4e87-a1d0-993056e32f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:47.404706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 869/1737] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:48:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 646 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:47.749096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9704a4-2237-465e-b4fd-46b91023136f service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:47.752137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2d9704a4-2237-465e-b4fd-46b91023136f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:47.877263 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88a39-e18a-4cda-a312-d597f08be3aa None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.877803 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88a39-e18a-4cda-a312-d597f08be3aa None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.877890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88a39-e18a-4cda-a312-d597f08be3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.965319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0ec88a39-e18a-4cda-a312-d597f08be3aa None None] MFA Rules not processed for user `b6ca4276e1554d4184451313b2e09a88`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:47.975756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0ec88a39-e18a-4cda-a312-d597f08be3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:48.153991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 870/1738] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 31 09:48:48.185616 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9704a4-2237-465e-b4fd-46b91023136f service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.218050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c3867-747b-4520-9d1a-17bb65cacd80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.219637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c3867-747b-4520-9d1a-17bb65cacd80 service swift] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.220203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c3867-747b-4520-9d1a-17bb65cacd80 service swift] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.221175 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c3867-747b-4520-9d1a-17bb65cacd80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.258850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c3867-747b-4520-9d1a-17bb65cacd80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.269956 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c3867-747b-4520-9d1a-17bb65cacd80 service swift] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.294264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 869/1739] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:48:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:48:50.814260 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b61e02-daf0-4836-8590-d417fa07e7b5 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:50.815193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b61e02-daf0-4836-8590-d417fa07e7b5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:50.815193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b61e02-daf0-4836-8590-d417fa07e7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:50.904810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-98b61e02-daf0-4836-8590-d417fa07e7b5 None None] MFA Rules not processed for user `77b5de46807b4c2ba9eb6a874da14b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:50.906351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-98b61e02-daf0-4836-8590-d417fa07e7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:51.030186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:51.271526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 871/1740] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:50 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:51.278004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c3867-747b-4520-9d1a-17bb65cacd80 service swift] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:51.287157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:51.500439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c3867-747b-4520-9d1a-17bb65cacd80 service swift] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:51.517072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833a042-37c6-460b-acfd-3041464ffb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:51.519413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833a042-37c6-460b-acfd-3041464ffb32 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:51.521446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833a042-37c6-460b-acfd-3041464ffb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:51.524018 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833a042-37c6-460b-acfd-3041464ffb32 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:51.525171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833a042-37c6-460b-acfd-3041464ffb32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:51.530384 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833a042-37c6-460b-acfd-3041464ffb32 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:51.621152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 870/1741] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:51 2022] POST /identity/v3/projects => generated 403 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:51.627474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b61e02-daf0-4836-8590-d417fa07e7b5 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:51.644133 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b61e02-daf0-4836-8590-d417fa07e7b5 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:51.808535 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd19eff-05ce-40a5-b5cc-726db58afee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:51.809177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd19eff-05ce-40a5-b5cc-726db58afee9 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:51.809736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd19eff-05ce-40a5-b5cc-726db58afee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:51.810107 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd19eff-05ce-40a5-b5cc-726db58afee9 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:51.814780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd19eff-05ce-40a5-b5cc-726db58afee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:51.819663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd19eff-05ce-40a5-b5cc-726db58afee9 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:51.872267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 872/1742] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:51 2022] POST /identity/v3/users => generated 308 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:51.874561 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833a042-37c6-460b-acfd-3041464ffb32 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:51.897303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833a042-37c6-460b-acfd-3041464ffb32 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:51.908682 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c743d8c-0e40-4374-82b0-bf1d10be8c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:51.909912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743d8c-0e40-4374-82b0-bf1d10be8c1a admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:51.910359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743d8c-0e40-4374-82b0-bf1d10be8c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:51.910950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743d8c-0e40-4374-82b0-bf1d10be8c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:51.914218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c743d8c-0e40-4374-82b0-bf1d10be8c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:51.917680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c743d8c-0e40-4374-82b0-bf1d10be8c1a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:51.935786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 871/1743] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:51.938596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd19eff-05ce-40a5-b5cc-726db58afee9 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:51.954738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd19eff-05ce-40a5-b5cc-726db58afee9 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:51.972815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdab13-8f71-4ea2-845a-1629202244ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:51.976003 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdab13-8f71-4ea2-845a-1629202244ed admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:51.977089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdab13-8f71-4ea2-845a-1629202244ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:51.977760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdab13-8f71-4ea2-845a-1629202244ed admin admin] PATH_INFO: `/v3/projects/2c95656e3c134a73824dbf6870fb10f5/users/06d238ac37dc46dd8b902c725e808666/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:52.051367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdab13-8f71-4ea2-845a-1629202244ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c95656e3c134a73824dbf6870fb10f5, user_id=06d238ac37dc46dd8b902c725e808666, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:52.072478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdab13-8f71-4ea2-845a-1629202244ed admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:52.141887 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 873/1744] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:51 2022] PUT /identity/v3/projects/2c95656e3c134a73824dbf6870fb10f5/users/06d238ac37dc46dd8b902c725e808666/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:52.144763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c743d8c-0e40-4374-82b0-bf1d10be8c1a admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:52.181359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c743d8c-0e40-4374-82b0-bf1d10be8c1a admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:52.444293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab12f2-285f-4765-8cf9-83ea2e6fd4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:52.446967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab12f2-285f-4765-8cf9-83ea2e6fd4da admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:52.449865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab12f2-285f-4765-8cf9-83ea2e6fd4da admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:52.452164 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab12f2-285f-4765-8cf9-83ea2e6fd4da admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:52.455440 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab12f2-285f-4765-8cf9-83ea2e6fd4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:52.470816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab12f2-285f-4765-8cf9-83ea2e6fd4da admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:52.512070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 872/1745] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:52.512169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdab13-8f71-4ea2-845a-1629202244ed admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:52.529517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdab13-8f71-4ea2-845a-1629202244ed admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:52.539968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4611e76-537d-4a97-aae5-02bc614ce9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:52.547082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4611e76-537d-4a97-aae5-02bc614ce9e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:52.547407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4611e76-537d-4a97-aae5-02bc614ce9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:52.547692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4611e76-537d-4a97-aae5-02bc614ce9e7 admin admin] PATH_INFO: `/v3/projects/2c95656e3c134a73824dbf6870fb10f5/users/06d238ac37dc46dd8b902c725e808666/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:52.555545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4611e76-537d-4a97-aae5-02bc614ce9e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c95656e3c134a73824dbf6870fb10f5, user_id=06d238ac37dc46dd8b902c725e808666, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:52.558360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4611e76-537d-4a97-aae5-02bc614ce9e7 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:52.582235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 874/1746] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:52 2022] PUT /identity/v3/projects/2c95656e3c134a73824dbf6870fb10f5/users/06d238ac37dc46dd8b902c725e808666/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:52.868450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc05d3-26b3-4725-bcde-93875f364d33 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:52.868983 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc05d3-26b3-4725-bcde-93875f364d33 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:52.869491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc05d3-26b3-4725-bcde-93875f364d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:52.939773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-79fc05d3-26b3-4725-bcde-93875f364d33 None None] MFA Rules not processed for user `980ab18eb5c646378d738079621a0dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:52.941909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-79fc05d3-26b3-4725-bcde-93875f364d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:53.359343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 873/1747] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:53.538616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8073d-3b73-4c26-ade2-5beb29b342e0 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:53.539380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8073d-3b73-4c26-ade2-5beb29b342e0 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:53.539919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8073d-3b73-4c26-ade2-5beb29b342e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:53.739636 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e2c8073d-3b73-4c26-ade2-5beb29b342e0 None None] MFA Rules not processed for user `6ff1177119cb48e0a88b0f11a6b1b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:53.744123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e2c8073d-3b73-4c26-ade2-5beb29b342e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:54.007645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 875/1748] 10.222.0.71 () {58 vars in 1015 bytes} [Wed Aug 31 09:48:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:54.068392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc05d3-26b3-4725-bcde-93875f364d33 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:54.072631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-79fc05d3-26b3-4725-bcde-93875f364d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:54.080738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8073d-3b73-4c26-ade2-5beb29b342e0 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:54.093792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8073d-3b73-4c26-ade2-5beb29b342e0 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:54.100880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52cb9a-a224-4864-bc07-39645406ebee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:54.102021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52cb9a-a224-4864-bc07-39645406ebee service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.102564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52cb9a-a224-4864-bc07-39645406ebee service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.103077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52cb9a-a224-4864-bc07-39645406ebee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.105052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc05d3-26b3-4725-bcde-93875f364d33 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:54.115742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cac938-9bc1-4d63-9fc1-3883b343c28f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:54.116680 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52cb9a-a224-4864-bc07-39645406ebee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:54.116909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cac938-9bc1-4d63-9fc1-3883b343c28f service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.117879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cac938-9bc1-4d63-9fc1-3883b343c28f service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.118585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cac938-9bc1-4d63-9fc1-3883b343c28f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.119073 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52cb9a-a224-4864-bc07-39645406ebee service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:54.136271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 876/1749] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:54.144072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cac938-9bc1-4d63-9fc1-3883b343c28f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:54.152303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cac938-9bc1-4d63-9fc1-3883b343c28f service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:54.234614 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 874/1750] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:54.451522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52cb9a-a224-4864-bc07-39645406ebee service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:54.471074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:54.684070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52cb9a-a224-4864-bc07-39645406ebee service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:54.700332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:54.702166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.703457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.705561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] PATH_INFO: `/v3/users/47aecbc7a7e146d6ad996d69de0d7beb` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.708706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47aecbc7a7e146d6ad996d69de0d7beb)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:54.726976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:54.938205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47aecbc7a7e146d6ad996d69de0d7beb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:54.994265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47aecbc7a7e146d6ad996d69de0d7beb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:54.997079 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47aecbc7a7e146d6ad996d69de0d7beb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:55.000820 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47aecbc7a7e146d6ad996d69de0d7beb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:55.029923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47aecbc7a7e146d6ad996d69de0d7beb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:55.102317 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 877/1751] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:54 2022] DELETE /identity/v3/users/47aecbc7a7e146d6ad996d69de0d7beb => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:55.109777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba3a2d-a4c7-48e2-8a24-ef2361158c16 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.110413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba3a2d-a4c7-48e2-8a24-ef2361158c16 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.110797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba3a2d-a4c7-48e2-8a24-ef2361158c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.163114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:55.236668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-47ba3a2d-a4c7-48e2-8a24-ef2361158c16 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:55.238586 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-47ba3a2d-a4c7-48e2-8a24-ef2361158c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:55.556221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 875/1752] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:56.037021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:56.042439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:56.403224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e444f-0a39-44f5-af45-4b7270ff3545 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:56.423907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0468e-551a-44d2-ba19-9db3a37a3f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:56.426249 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0468e-551a-44d2-ba19-9db3a37a3f84 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:56.429270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0468e-551a-44d2-ba19-9db3a37a3f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:56.430549 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0468e-551a-44d2-ba19-9db3a37a3f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:56.489641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0468e-551a-44d2-ba19-9db3a37a3f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:56.492991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0468e-551a-44d2-ba19-9db3a37a3f84 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:56.549193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 878/1753] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:57.586570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb38fc5-cea4-4798-af03-1461398a43fa None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:57.588011 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb38fc5-cea4-4798-af03-1461398a43fa None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:57.588694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb38fc5-cea4-4798-af03-1461398a43fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:57.761397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-7bb38fc5-cea4-4798-af03-1461398a43fa None None] MFA Rules not processed for user `06d238ac37dc46dd8b902c725e808666`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:57.764877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7bb38fc5-cea4-4798-af03-1461398a43fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:57.937848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 876/1754] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:48:57 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:58.803962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0468e-551a-44d2-ba19-9db3a37a3f84 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:58.809026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-44e0468e-551a-44d2-ba19-9db3a37a3f84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:58.876290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0468e-551a-44d2-ba19-9db3a37a3f84 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:58.887542 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:58.889699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:58.889980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:58.892083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] PATH_INFO: `/v3/projects/5131f516ee1c473fbac6c74b1d73ddcc` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:58.918447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5131f516ee1c473fbac6c74b1d73ddcc)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:58.932518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:58.998365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:59.015646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc4127-eae7-4731-baff-6c0a4a932bf6 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:59.017445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc4127-eae7-4731-baff-6c0a4a932bf6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:59.017975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc4127-eae7-4731-baff-6c0a4a932bf6 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:59.020911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 877/1755] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 09:48:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:59.039953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5131f516ee1c473fbac6c74b1d73ddcc', 'request_id': 'req-d318e85a-92e4-4afa-b15f-2f047fdecf9d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.054289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5131f516ee1c473fbac6c74b1d73ddcc', 'request_id': 'req-d318e85a-92e4-4afa-b15f-2f047fdecf9d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.068646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5131f516ee1c473fbac6c74b1d73ddcc', 'request_id': 'req-d318e85a-92e4-4afa-b15f-2f047fdecf9d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.073051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda10a7-663e-46e1-b5e6-b2eeec095725 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:59.073634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda10a7-663e-46e1-b5e6-b2eeec095725 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:59.074056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda10a7-663e-46e1-b5e6-b2eeec095725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:59.093573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] The token cache is being invalidate because project 5131f516ee1c473fbac6c74b1d73ddcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:59.094662 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.096972 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.106326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 879/1756] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:58 2022] DELETE /identity/v3/projects/5131f516ee1c473fbac6c74b1d73ddcc => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:59.109380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:59.112443 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:59.132761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-2bda10a7-663e-46e1-b5e6-b2eeec095725 None None] MFA Rules not processed for user `ef72bb95992c4a21ae2dbb2a08c51421`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:59.134409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2bda10a7-663e-46e1-b5e6-b2eeec095725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:59.278538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 878/1757] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 09:48:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:59.292137 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318e85a-92e4-4afa-b15f-2f047fdecf9d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:59.322953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700effc-7067-4042-b21e-ee4ff4fcb422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:59.324994 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700effc-7067-4042-b21e-ee4ff4fcb422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:59.325898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700effc-7067-4042-b21e-ee4ff4fcb422 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:59.326588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700effc-7067-4042-b21e-ee4ff4fcb422 admin admin] PATH_INFO: `/v3/users/ead51d4314f446ccbe0b4d626d424a64` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:59.329309 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700effc-7067-4042-b21e-ee4ff4fcb422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ead51d4314f446ccbe0b4d626d424a64)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:59.335030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700effc-7067-4042-b21e-ee4ff4fcb422 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:59.471983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5700effc-7067-4042-b21e-ee4ff4fcb422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ead51d4314f446ccbe0b4d626d424a64'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.478759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda10a7-663e-46e1-b5e6-b2eeec095725 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:59.480430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2bda10a7-663e-46e1-b5e6-b2eeec095725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:59.497985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda10a7-663e-46e1-b5e6-b2eeec095725 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:59.502087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5700effc-7067-4042-b21e-ee4ff4fcb422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ead51d4314f446ccbe0b4d626d424a64'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.503098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5700effc-7067-4042-b21e-ee4ff4fcb422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ead51d4314f446ccbe0b4d626d424a64'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.506738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5700effc-7067-4042-b21e-ee4ff4fcb422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ead51d4314f446ccbe0b4d626d424a64'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.528338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60f334-35ef-4076-a579-60e005d6a359 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:59.529915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60f334-35ef-4076-a579-60e005d6a359 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:59.530590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60f334-35ef-4076-a579-60e005d6a359 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:59.531304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60f334-35ef-4076-a579-60e005d6a359 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:59.533767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5700effc-7067-4042-b21e-ee4ff4fcb422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ead51d4314f446ccbe0b4d626d424a64'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.544544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-da60f334-35ef-4076-a579-60e005d6a359 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:59.564570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60f334-35ef-4076-a579-60e005d6a359 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:59.566397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60f334-35ef-4076-a579-60e005d6a359 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:59.571597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 880/1758] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:59 2022] DELETE /identity/v3/users/ead51d4314f446ccbe0b4d626d424a64 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:59.579885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 879/1759] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:00.203784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e81516-9f7b-4949-b008-5ddcf34a68d5 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:00.204332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e81516-9f7b-4949-b008-5ddcf34a68d5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:00.204805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e81516-9f7b-4949-b008-5ddcf34a68d5 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:00.207809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 881/1760] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 09:49:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:49:00.326914 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad149914-643f-46a1-afd6-9c0929be6717 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:00.327615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad149914-643f-46a1-afd6-9c0929be6717 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:00.327615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad149914-643f-46a1-afd6-9c0929be6717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:00.380611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ad149914-643f-46a1-afd6-9c0929be6717 None None] MFA Rules not processed for user `560df3eb05f14b6489158ab053f282ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:00.382054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ad149914-643f-46a1-afd6-9c0929be6717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:00.395789 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:00.437329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e81516-9f7b-4949-b008-5ddcf34a68d5 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:00.440303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-27e81516-9f7b-4949-b008-5ddcf34a68d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:00.674788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:00.729178 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 880/1761] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 09:49:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:00.790528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad149914-643f-46a1-afd6-9c0929be6717 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:00.792189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ad149914-643f-46a1-afd6-9c0929be6717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:00.820009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad149914-643f-46a1-afd6-9c0929be6717 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:00.832467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e81516-9f7b-4949-b008-5ddcf34a68d5 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:00.842098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3715b-a907-420b-8b73-9e7c60e27077 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:00.843031 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3715b-a907-420b-8b73-9e7c60e27077 service placement] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:00.843843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3715b-a907-420b-8b73-9e7c60e27077 service placement] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:00.844620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3715b-a907-420b-8b73-9e7c60e27077 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:00.867494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3715b-a907-420b-8b73-9e7c60e27077 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:00.869386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3715b-a907-420b-8b73-9e7c60e27077 service placement] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:00.877486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8439a-0371-41c4-b8da-89f00f6853a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:00.882620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8439a-0371-41c4-b8da-89f00f6853a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:00.884114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8439a-0371-41c4-b8da-89f00f6853a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:00.884956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8439a-0371-41c4-b8da-89f00f6853a8 admin admin] PATH_INFO: `/v3/projects/c50ca06f92084817ba5c30386532c15d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:00.897829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8439a-0371-41c4-b8da-89f00f6853a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c50ca06f92084817ba5c30386532c15d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:00.900556 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 881/1762] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:00.911638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8439a-0371-41c4-b8da-89f00f6853a8 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:01.225720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0db8439a-0371-41c4-b8da-89f00f6853a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c50ca06f92084817ba5c30386532c15d', 'request_id': 'req-0db8439a-0371-41c4-b8da-89f00f6853a8'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:01.268752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0db8439a-0371-41c4-b8da-89f00f6853a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c50ca06f92084817ba5c30386532c15d', 'request_id': 'req-0db8439a-0371-41c4-b8da-89f00f6853a8'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:01.288612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0db8439a-0371-41c4-b8da-89f00f6853a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c50ca06f92084817ba5c30386532c15d', 'request_id': 'req-0db8439a-0371-41c4-b8da-89f00f6853a8'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:01.352019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0db8439a-0371-41c4-b8da-89f00f6853a8 admin admin] The token cache is being invalidate because project c50ca06f92084817ba5c30386532c15d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:01.353917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0db8439a-0371-41c4-b8da-89f00f6853a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:01.355769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0db8439a-0371-41c4-b8da-89f00f6853a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:01.369327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 882/1763] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:00 2022] DELETE /identity/v3/projects/c50ca06f92084817ba5c30386532c15d => generated 0 bytes in 932 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:01.376772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4802f-0ee6-4455-9602-5126afb68243 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.377691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4802f-0ee6-4455-9602-5126afb68243 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.378575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4802f-0ee6-4455-9602-5126afb68243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.447544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-23b4802f-0ee6-4455-9602-5126afb68243 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:01.457678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-23b4802f-0ee6-4455-9602-5126afb68243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:01.631753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 882/1764] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:01.640895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11530ab6-8c3a-40f8-aab7-c33307b04385 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.641366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11530ab6-8c3a-40f8-aab7-c33307b04385 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.642050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11530ab6-8c3a-40f8-aab7-c33307b04385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.674254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-11530ab6-8c3a-40f8-aab7-c33307b04385 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:01.677569 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-11530ab6-8c3a-40f8-aab7-c33307b04385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:01.696804 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 883/1765] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:01.703574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4802f-0ee6-4455-9602-5126afb68243 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:01.731960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4802f-0ee6-4455-9602-5126afb68243 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:01.741902 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8961a-386b-47c4-81c6-cd5880ca96b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:01.743018 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8961a-386b-47c4-81c6-cd5880ca96b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.743248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8961a-386b-47c4-81c6-cd5880ca96b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.744481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8961a-386b-47c4-81c6-cd5880ca96b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.745733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8961a-386b-47c4-81c6-cd5880ca96b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:01.746403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8961a-386b-47c4-81c6-cd5880ca96b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:01.750947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8961a-386b-47c4-81c6-cd5880ca96b4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:01.779943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 883/1766] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:01.787730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11530ab6-8c3a-40f8-aab7-c33307b04385 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:01.809572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11530ab6-8c3a-40f8-aab7-c33307b04385 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:01.819313 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45316a3d-631a-40b3-a148-9be1e47feff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:01.819954 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45316a3d-631a-40b3-a148-9be1e47feff2 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.820241 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45316a3d-631a-40b3-a148-9be1e47feff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.820495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45316a3d-631a-40b3-a148-9be1e47feff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.821387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45316a3d-631a-40b3-a148-9be1e47feff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:01.823884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45316a3d-631a-40b3-a148-9be1e47feff2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:01.883765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 884/1767] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:01 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:01.886142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8961a-386b-47c4-81c6-cd5880ca96b4 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:01.911867 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8961a-386b-47c4-81c6-cd5880ca96b4 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.030718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9c0f1-3204-46b8-a81c-ecad05a95e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.031295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9c0f1-3204-46b8-a81c-ecad05a95e1a admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.031825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9c0f1-3204-46b8-a81c-ecad05a95e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.032106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9c0f1-3204-46b8-a81c-ecad05a95e1a admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.032777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9c0f1-3204-46b8-a81c-ecad05a95e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.034896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9c0f1-3204-46b8-a81c-ecad05a95e1a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.075899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 884/1768] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:01 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:02.082551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45316a3d-631a-40b3-a148-9be1e47feff2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.104725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45316a3d-631a-40b3-a148-9be1e47feff2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.115097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cb8d5-51f6-42fd-b6ad-c948aa34c6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.115776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cb8d5-51f6-42fd-b6ad-c948aa34c6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.116254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cb8d5-51f6-42fd-b6ad-c948aa34c6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.116822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cb8d5-51f6-42fd-b6ad-c948aa34c6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.118065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cb8d5-51f6-42fd-b6ad-c948aa34c6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.122377 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cb8d5-51f6-42fd-b6ad-c948aa34c6e4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.139612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 885/1769] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:02.144415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9c0f1-3204-46b8-a81c-ecad05a95e1a admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.156741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9c0f1-3204-46b8-a81c-ecad05a95e1a admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.162449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa7f8e-e2b0-4f4a-928d-30b382ffb923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.163175 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa7f8e-e2b0-4f4a-928d-30b382ffb923 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.163525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa7f8e-e2b0-4f4a-928d-30b382ffb923 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.163988 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa7f8e-e2b0-4f4a-928d-30b382ffb923 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.164738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa7f8e-e2b0-4f4a-928d-30b382ffb923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.166438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa7f8e-e2b0-4f4a-928d-30b382ffb923 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.169105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:02.184188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 885/1770] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:02.185420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cb8d5-51f6-42fd-b6ad-c948aa34c6e4 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.204583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cb8d5-51f6-42fd-b6ad-c948aa34c6e4 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.213128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a5eac-86d3-4af9-b367-37712162cecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.214074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a5eac-86d3-4af9-b367-37712162cecb admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.214584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a5eac-86d3-4af9-b367-37712162cecb admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.215226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a5eac-86d3-4af9-b367-37712162cecb admin admin] PATH_INFO: `/v3/projects/070571af27184314b2191fc2a3a1dd37/users/08e074f594bf4a5d8ed37a9d5cb53255/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.238278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a5eac-86d3-4af9-b367-37712162cecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=070571af27184314b2191fc2a3a1dd37, user_id=08e074f594bf4a5d8ed37a9d5cb53255, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.242086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a5eac-86d3-4af9-b367-37712162cecb admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.264789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 886/1771] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:02 2022] PUT /identity/v3/projects/070571af27184314b2191fc2a3a1dd37/users/08e074f594bf4a5d8ed37a9d5cb53255/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:02.280097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157da7f6-6cdd-4877-b2ec-2521067ad95a None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.280716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157da7f6-6cdd-4877-b2ec-2521067ad95a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.281505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157da7f6-6cdd-4877-b2ec-2521067ad95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.342087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-157da7f6-6cdd-4877-b2ec-2521067ad95a None None] MFA Rules not processed for user `08e074f594bf4a5d8ed37a9d5cb53255`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:02.345782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-157da7f6-6cdd-4877-b2ec-2521067ad95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:02.463660 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 886/1772] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:02.470732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a5eac-86d3-4af9-b367-37712162cecb admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.578934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a5eac-86d3-4af9-b367-37712162cecb admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.585034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01999471-ab39-40e1-afe8-c666ce165f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.585827 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01999471-ab39-40e1-afe8-c666ce165f71 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.586182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01999471-ab39-40e1-afe8-c666ce165f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.586529 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01999471-ab39-40e1-afe8-c666ce165f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.587420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01999471-ab39-40e1-afe8-c666ce165f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.589711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01999471-ab39-40e1-afe8-c666ce165f71 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.596835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:02.659551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 887/1773] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:02 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:02.666390 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157da7f6-6cdd-4877-b2ec-2521067ad95a None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.696285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157da7f6-6cdd-4877-b2ec-2521067ad95a None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.834727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633f993-b37d-43e7-9364-c20b8e9fbbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.835807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633f993-b37d-43e7-9364-c20b8e9fbbbf admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.836276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633f993-b37d-43e7-9364-c20b8e9fbbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.836737 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633f993-b37d-43e7-9364-c20b8e9fbbbf admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.837905 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633f993-b37d-43e7-9364-c20b8e9fbbbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.842098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633f993-b37d-43e7-9364-c20b8e9fbbbf admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.882562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 887/1774] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:02 2022] POST /identity/v3/users => generated 299 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:02.889077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01999471-ab39-40e1-afe8-c666ce165f71 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.908808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01999471-ab39-40e1-afe8-c666ce165f71 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.918424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614856d-df49-45be-9d18-474b07db2111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.919191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614856d-df49-45be-9d18-474b07db2111 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.919762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614856d-df49-45be-9d18-474b07db2111 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.920335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614856d-df49-45be-9d18-474b07db2111 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.922404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614856d-df49-45be-9d18-474b07db2111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.927653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614856d-df49-45be-9d18-474b07db2111 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.962366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 888/1775] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:02.970362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633f993-b37d-43e7-9364-c20b8e9fbbbf admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.995654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633f993-b37d-43e7-9364-c20b8e9fbbbf admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.002324 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b372555-574e-43d8-a9c4-4226a442d83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.003088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b372555-574e-43d8-a9c4-4226a442d83d admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.003842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b372555-574e-43d8-a9c4-4226a442d83d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.004151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b372555-574e-43d8-a9c4-4226a442d83d admin admin] PATH_INFO: `/v3/projects/95f67939e55745d087a2f1fb21bf7f03/users/b36547d7dc1a489bb2268bbdd6d9c65b/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.062579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b372555-574e-43d8-a9c4-4226a442d83d admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f67939e55745d087a2f1fb21bf7f03, user_id=b36547d7dc1a489bb2268bbdd6d9c65b, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.073438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b372555-574e-43d8-a9c4-4226a442d83d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.118237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 888/1776] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:02 2022] PUT /identity/v3/projects/95f67939e55745d087a2f1fb21bf7f03/users/b36547d7dc1a489bb2268bbdd6d9c65b/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:03.123223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614856d-df49-45be-9d18-474b07db2111 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.163365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614856d-df49-45be-9d18-474b07db2111 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.303218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e35c7-e2d8-4991-915f-916b12a8c96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.305360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e35c7-e2d8-4991-915f-916b12a8c96c admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.305817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e35c7-e2d8-4991-915f-916b12a8c96c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.306241 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e35c7-e2d8-4991-915f-916b12a8c96c admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.307500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e35c7-e2d8-4991-915f-916b12a8c96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.315442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e35c7-e2d8-4991-915f-916b12a8c96c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.333512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b372555-574e-43d8-a9c4-4226a442d83d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.334950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 889/1777] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:03.350335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b372555-574e-43d8-a9c4-4226a442d83d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.354601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d095f-993d-482e-945a-5afe5f649218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.355343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d095f-993d-482e-945a-5afe5f649218 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.355626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d095f-993d-482e-945a-5afe5f649218 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.355907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d095f-993d-482e-945a-5afe5f649218 admin admin] PATH_INFO: `/v3/projects/95f67939e55745d087a2f1fb21bf7f03/users/b36547d7dc1a489bb2268bbdd6d9c65b/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.385281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47522c8-30d7-414c-baf3-f3dbf3624ee3 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.385833 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47522c8-30d7-414c-baf3-f3dbf3624ee3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.386108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d095f-993d-482e-945a-5afe5f649218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f67939e55745d087a2f1fb21bf7f03, user_id=b36547d7dc1a489bb2268bbdd6d9c65b, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.386725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47522c8-30d7-414c-baf3-f3dbf3624ee3 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.389270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 890/1778] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 09:49:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:49:03.402706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d095f-993d-482e-945a-5afe5f649218 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.405795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a995636-7168-41e1-beb8-5d6ed882f210 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.406074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a995636-7168-41e1-beb8-5d6ed882f210 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.406350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a995636-7168-41e1-beb8-5d6ed882f210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.450165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 889/1779] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:03 2022] PUT /identity/v3/projects/95f67939e55745d087a2f1fb21bf7f03/users/b36547d7dc1a489bb2268bbdd6d9c65b/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:03.465621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-7a995636-7168-41e1-beb8-5d6ed882f210 None None] MFA Rules not processed for user `560df3eb05f14b6489158ab053f282ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:03.467348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7a995636-7168-41e1-beb8-5d6ed882f210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.468201 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2310c5f-f56c-48d6-8359-fcf6faa7702a None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.469261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2310c5f-f56c-48d6-8359-fcf6faa7702a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.469750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2310c5f-f56c-48d6-8359-fcf6faa7702a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.510655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b2310c5f-f56c-48d6-8359-fcf6faa7702a None None] MFA Rules not processed for user `b36547d7dc1a489bb2268bbdd6d9c65b`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:03.512637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b2310c5f-f56c-48d6-8359-fcf6faa7702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.588976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:03.602995 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 891/1780] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 09:49:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:03.604073 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a995636-7168-41e1-beb8-5d6ed882f210 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.605716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7a995636-7168-41e1-beb8-5d6ed882f210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.729738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 890/1781] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:03.729738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2310c5f-f56c-48d6-8359-fcf6faa7702a None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.730910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b2310c5f-f56c-48d6-8359-fcf6faa7702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.769459 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2310c5f-f56c-48d6-8359-fcf6faa7702a None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.793155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6322a80-bf0c-4943-9ded-cf67c9f0d4b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.794034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6322a80-bf0c-4943-9ded-cf67c9f0d4b2 service placement] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.794552 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6322a80-bf0c-4943-9ded-cf67c9f0d4b2 service placement] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.795271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6322a80-bf0c-4943-9ded-cf67c9f0d4b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.795385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a995636-7168-41e1-beb8-5d6ed882f210 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.804662 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3753949-491b-4fd0-b702-a060523f2e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.805499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3753949-491b-4fd0-b702-a060523f2e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.805789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3753949-491b-4fd0-b702-a060523f2e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.806613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3753949-491b-4fd0-b702-a060523f2e56 admin admin] PATH_INFO: `/v3/users/77b5de46807b4c2ba9eb6a874da14b2b` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.808925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3753949-491b-4fd0-b702-a060523f2e56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b5de46807b4c2ba9eb6a874da14b2b)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.810933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3753949-491b-4fd0-b702-a060523f2e56 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.819150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6322a80-bf0c-4943-9ded-cf67c9f0d4b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.820785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6322a80-bf0c-4943-9ded-cf67c9f0d4b2 service placement] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.834021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:03.852155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 891/1782] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:03.873680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10f82d-5039-4e5c-993d-4693bedd06b6 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.874707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10f82d-5039-4e5c-993d-4693bedd06b6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.874707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10f82d-5039-4e5c-993d-4693bedd06b6 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.880156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 892/1783] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 09:49:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:49:03.881088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10f82d-5039-4e5c-993d-4693bedd06b6 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.886202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fa10f82d-5039-4e5c-993d-4693bedd06b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:04.010807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e3753949-491b-4fd0-b702-a060523f2e56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b5de46807b4c2ba9eb6a874da14b2b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.059409 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e3753949-491b-4fd0-b702-a060523f2e56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b5de46807b4c2ba9eb6a874da14b2b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.060562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e3753949-491b-4fd0-b702-a060523f2e56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b5de46807b4c2ba9eb6a874da14b2b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.061446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e3753949-491b-4fd0-b702-a060523f2e56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b5de46807b4c2ba9eb6a874da14b2b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.086987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e3753949-491b-4fd0-b702-a060523f2e56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b5de46807b4c2ba9eb6a874da14b2b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.115731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10f82d-5039-4e5c-993d-4693bedd06b6 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.131664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 892/1784] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:03 2022] DELETE /identity/v3/users/77b5de46807b4c2ba9eb6a874da14b2b => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:04.131964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b7a96-9c11-442f-a292-3322b522ac0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.131964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b7a96-9c11-442f-a292-3322b522ac0e service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.132150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b7a96-9c11-442f-a292-3322b522ac0e service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.132460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b7a96-9c11-442f-a292-3322b522ac0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.134033 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc2bd9-598a-4c0c-9347-03f907932a2a None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.134322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc2bd9-598a-4c0c-9347-03f907932a2a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.134603 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc2bd9-598a-4c0c-9347-03f907932a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.139434 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-408b7a96-9c11-442f-a292-3322b522ac0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:04.188119 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b7a96-9c11-442f-a292-3322b522ac0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.192691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b7a96-9c11-442f-a292-3322b522ac0e service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.213868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-29bc2bd9-598a-4c0c-9347-03f907932a2a None None] MFA Rules not processed for user `ef72bb95992c4a21ae2dbb2a08c51421`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:04.215199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-29bc2bd9-598a-4c0c-9347-03f907932a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:04.234043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 893/1785] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:04.534062 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 893/1786] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 09:49:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:04.570299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b7a96-9c11-442f-a292-3322b522ac0e service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.571874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-408b7a96-9c11-442f-a292-3322b522ac0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:04.721885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b7a96-9c11-442f-a292-3322b522ac0e service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.731931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572b35e-ac2e-4b4c-a4d6-ba2e2b9eca1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.733292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572b35e-ac2e-4b4c-a4d6-ba2e2b9eca1e service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.737051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572b35e-ac2e-4b4c-a4d6-ba2e2b9eca1e service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.737051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572b35e-ac2e-4b4c-a4d6-ba2e2b9eca1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.755545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc2bd9-598a-4c0c-9347-03f907932a2a None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.756520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572b35e-ac2e-4b4c-a4d6-ba2e2b9eca1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.757992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-29bc2bd9-598a-4c0c-9347-03f907932a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:04.760235 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572b35e-ac2e-4b4c-a4d6-ba2e2b9eca1e service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.800717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 894/1787] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:04.868589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc2bd9-598a-4c0c-9347-03f907932a2a None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.873861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.874594 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.874955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.875276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] PATH_INFO: `/v3/projects/742f1c1cfb7c4116b40bacb83e7a43dd` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.876791 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=742f1c1cfb7c4116b40bacb83e7a43dd)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.880227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.912176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:04.963265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '742f1c1cfb7c4116b40bacb83e7a43dd', 'request_id': 'req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.976100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '742f1c1cfb7c4116b40bacb83e7a43dd', 'request_id': 'req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.986798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '742f1c1cfb7c4116b40bacb83e7a43dd', 'request_id': 'req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.003836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] The token cache is being invalidate because project 742f1c1cfb7c4116b40bacb83e7a43dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:05.004306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.006074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.008536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 894/1788] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:04 2022] DELETE /identity/v3/projects/742f1c1cfb7c4116b40bacb83e7a43dd => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:05.012936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572b35e-ac2e-4b4c-a4d6-ba2e2b9eca1e service nova] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.031163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572b35e-ac2e-4b4c-a4d6-ba2e2b9eca1e service nova] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.116402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:05.149137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.150427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.151049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.151722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] PATH_INFO: `/v3/users/06d238ac37dc46dd8b902c725e808666` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.153919 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d238ac37dc46dd8b902c725e808666)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.157958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.229848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d238ac37dc46dd8b902c725e808666'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.241305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d238ac37dc46dd8b902c725e808666'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.241985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d238ac37dc46dd8b902c725e808666'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.263582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d238ac37dc46dd8b902c725e808666'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.264539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d238ac37dc46dd8b902c725e808666'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.278459 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 895/1789] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:05 2022] DELETE /identity/v3/users/06d238ac37dc46dd8b902c725e808666 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:05.388490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.390346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.405129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33233a-0d40-4784-bbd2-4e629f0ae7c6 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.508769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5b9d9-4134-436a-8b53-1654c7f48d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.509536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5b9d9-4134-436a-8b53-1654c7f48d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.509885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5b9d9-4134-436a-8b53-1654c7f48d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.510248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5b9d9-4134-436a-8b53-1654c7f48d62 admin admin] PATH_INFO: `/v3/users/08e074f594bf4a5d8ed37a9d5cb53255` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.515547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5b9d9-4134-436a-8b53-1654c7f48d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e074f594bf4a5d8ed37a9d5cb53255)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.517575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5b9d9-4134-436a-8b53-1654c7f48d62 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.576052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.578785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.593411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f3d5b9d9-4134-436a-8b53-1654c7f48d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e074f594bf4a5d8ed37a9d5cb53255'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.615090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f3d5b9d9-4134-436a-8b53-1654c7f48d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e074f594bf4a5d8ed37a9d5cb53255'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.616543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f3d5b9d9-4134-436a-8b53-1654c7f48d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e074f594bf4a5d8ed37a9d5cb53255'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.617292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f3d5b9d9-4134-436a-8b53-1654c7f48d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e074f594bf4a5d8ed37a9d5cb53255'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.628130 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f3d5b9d9-4134-436a-8b53-1654c7f48d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e074f594bf4a5d8ed37a9d5cb53255'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.650467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419e1ae-112d-41b0-a60b-b5a4dfcf308f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.653084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 895/1790] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:05 2022] DELETE /identity/v3/users/08e074f594bf4a5d8ed37a9d5cb53255 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:05.658719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd48cd-c11c-462c-bccc-f5245670c1ef None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.659012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd48cd-c11c-462c-bccc-f5245670c1ef None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.659317 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd48cd-c11c-462c-bccc-f5245670c1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.663047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691665d-1bb6-4ebd-a386-dc78f870a91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.664910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691665d-1bb6-4ebd-a386-dc78f870a91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.665600 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691665d-1bb6-4ebd-a386-dc78f870a91d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.666502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691665d-1bb6-4ebd-a386-dc78f870a91d admin admin] PATH_INFO: `/v3/projects/2c95656e3c134a73824dbf6870fb10f5` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.669193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691665d-1bb6-4ebd-a386-dc78f870a91d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c95656e3c134a73824dbf6870fb10f5)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.674887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691665d-1bb6-4ebd-a386-dc78f870a91d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.687915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-13bd48cd-c11c-462c-bccc-f5245670c1ef None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:05.689592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-13bd48cd-c11c-462c-bccc-f5245670c1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.726599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:05.774349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7691665d-1bb6-4ebd-a386-dc78f870a91d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c95656e3c134a73824dbf6870fb10f5', 'request_id': 'req-7691665d-1bb6-4ebd-a386-dc78f870a91d'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.786763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 896/1791] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:05.790067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7691665d-1bb6-4ebd-a386-dc78f870a91d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c95656e3c134a73824dbf6870fb10f5', 'request_id': 'req-7691665d-1bb6-4ebd-a386-dc78f870a91d'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.798265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd48cd-c11c-462c-bccc-f5245670c1ef None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.800048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-13bd48cd-c11c-462c-bccc-f5245670c1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.807738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7691665d-1bb6-4ebd-a386-dc78f870a91d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c95656e3c134a73824dbf6870fb10f5', 'request_id': 'req-7691665d-1bb6-4ebd-a386-dc78f870a91d'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.853239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7691665d-1bb6-4ebd-a386-dc78f870a91d admin admin] The token cache is being invalidate because project 2c95656e3c134a73824dbf6870fb10f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:05.854481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7691665d-1bb6-4ebd-a386-dc78f870a91d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.856234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7691665d-1bb6-4ebd-a386-dc78f870a91d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.867198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 896/1792] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:05 2022] DELETE /identity/v3/projects/2c95656e3c134a73824dbf6870fb10f5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:05.870637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c874e-0714-4174-b6d5-5b913e6892c4 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.871167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c874e-0714-4174-b6d5-5b913e6892c4 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.871655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c874e-0714-4174-b6d5-5b913e6892c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.932604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-e95c874e-0714-4174-b6d5-5b913e6892c4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:05.934516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e95c874e-0714-4174-b6d5-5b913e6892c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.944165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd48cd-c11c-462c-bccc-f5245670c1ef None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.026772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:06.050490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b7493-d4f1-40e5-b311-fa02fc0613ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.052072 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b7493-d4f1-40e5-b311-fa02fc0613ed service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.052921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b7493-d4f1-40e5-b311-fa02fc0613ed service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.053274 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b7493-d4f1-40e5-b311-fa02fc0613ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.056391 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4d8b7493-d4f1-40e5-b311-fa02fc0613ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.066417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 897/1793] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:06.078043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a02b6e-ed65-4aae-8576-630f27a56f0f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.078043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a02b6e-ed65-4aae-8576-630f27a56f0f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.078492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a02b6e-ed65-4aae-8576-630f27a56f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.081684 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b7493-d4f1-40e5-b311-fa02fc0613ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.084530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b7493-d4f1-40e5-b311-fa02fc0613ed service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.101106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-23a02b6e-ed65-4aae-8576-630f27a56f0f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:06.102374 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-23a02b6e-ed65-4aae-8576-630f27a56f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.106194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 897/1794] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:06.112530 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 898/1795] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:06.118214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b7493-d4f1-40e5-b311-fa02fc0613ed service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.134582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b7493-d4f1-40e5-b311-fa02fc0613ed service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.140821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162941f-b12e-429c-a81e-83d6f47f11cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.141659 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162941f-b12e-429c-a81e-83d6f47f11cc admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.142130 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162941f-b12e-429c-a81e-83d6f47f11cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.142678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162941f-b12e-429c-a81e-83d6f47f11cc admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.143641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162941f-b12e-429c-a81e-83d6f47f11cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:06.144044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162941f-b12e-429c-a81e-83d6f47f11cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.146374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162941f-b12e-429c-a81e-83d6f47f11cc admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.160978 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 898/1796] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:06.165185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a02b6e-ed65-4aae-8576-630f27a56f0f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.176037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a02b6e-ed65-4aae-8576-630f27a56f0f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.184016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323a2db-880b-424e-9104-76e38d97665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.184468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323a2db-880b-424e-9104-76e38d97665d admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.184747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323a2db-880b-424e-9104-76e38d97665d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.185025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323a2db-880b-424e-9104-76e38d97665d admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.185686 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323a2db-880b-424e-9104-76e38d97665d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.187883 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323a2db-880b-424e-9104-76e38d97665d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.254366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 899/1797] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:06 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:06.259811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162941f-b12e-429c-a81e-83d6f47f11cc admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.291027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162941f-b12e-429c-a81e-83d6f47f11cc admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.388074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1d68b-e4fe-4542-a13a-a6c542e9ab69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.388879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1d68b-e4fe-4542-a13a-a6c542e9ab69 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.389613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1d68b-e4fe-4542-a13a-a6c542e9ab69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.390104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1d68b-e4fe-4542-a13a-a6c542e9ab69 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.391393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1d68b-e4fe-4542-a13a-a6c542e9ab69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.394657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1d68b-e4fe-4542-a13a-a6c542e9ab69 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.448823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 899/1798] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:06 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:06.455035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323a2db-880b-424e-9104-76e38d97665d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.470612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323a2db-880b-424e-9104-76e38d97665d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.493815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ba4f6-ab29-4d3a-9181-95295250a0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.495215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ba4f6-ab29-4d3a-9181-95295250a0ba admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.496113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ba4f6-ab29-4d3a-9181-95295250a0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.496629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ba4f6-ab29-4d3a-9181-95295250a0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.503026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ba4f6-ab29-4d3a-9181-95295250a0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.507485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ba4f6-ab29-4d3a-9181-95295250a0ba admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.535622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 900/1799] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:06.538259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1d68b-e4fe-4542-a13a-a6c542e9ab69 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.551647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:06.573450 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1d68b-e4fe-4542-a13a-a6c542e9ab69 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.602728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25840be4-78b6-4498-8464-6ec7f5eb55a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.603684 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25840be4-78b6-4498-8464-6ec7f5eb55a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.604053 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25840be4-78b6-4498-8464-6ec7f5eb55a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.604485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25840be4-78b6-4498-8464-6ec7f5eb55a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.605554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25840be4-78b6-4498-8464-6ec7f5eb55a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.612772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25840be4-78b6-4498-8464-6ec7f5eb55a0 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.632013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 900/1800] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:06.638264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ba4f6-ab29-4d3a-9181-95295250a0ba admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.651085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ba4f6-ab29-4d3a-9181-95295250a0ba admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.667631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76436e17-9837-419e-b5f7-baa90034baca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.667631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436e17-9837-419e-b5f7-baa90034baca admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.667631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436e17-9837-419e-b5f7-baa90034baca admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.667631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436e17-9837-419e-b5f7-baa90034baca admin admin] PATH_INFO: `/v3/projects/0a987d86c86540bda66ee443f0b9a5af/users/7af77de115ae4d2fac7a3d2f52fe773b/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.700279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76436e17-9837-419e-b5f7-baa90034baca admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a987d86c86540bda66ee443f0b9a5af, user_id=7af77de115ae4d2fac7a3d2f52fe773b, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.706294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76436e17-9837-419e-b5f7-baa90034baca admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.788122 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 901/1801] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:06 2022] PUT /identity/v3/projects/0a987d86c86540bda66ee443f0b9a5af/users/7af77de115ae4d2fac7a3d2f52fe773b/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:06.798689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561438f2-9a4b-449a-b236-3132753e7b94 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.799355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561438f2-9a4b-449a-b236-3132753e7b94 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.799921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561438f2-9a4b-449a-b236-3132753e7b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.818712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76436e17-9837-419e-b5f7-baa90034baca admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.824754 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-76436e17-9837-419e-b5f7-baa90034baca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.839981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-561438f2-9a4b-449a-b236-3132753e7b94 None None] MFA Rules not processed for user `7af77de115ae4d2fac7a3d2f52fe773b`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:06.842228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-561438f2-9a4b-449a-b236-3132753e7b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.030702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 901/1802] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:07.037071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561438f2-9a4b-449a-b236-3132753e7b94 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.105245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76436e17-9837-419e-b5f7-baa90034baca admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.124113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.125211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.125691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.126152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] PATH_INFO: `/v3/projects/070571af27184314b2191fc2a3a1dd37` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.128017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=070571af27184314b2191fc2a3a1dd37)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.130731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.253836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '070571af27184314b2191fc2a3a1dd37', 'request_id': 'req-1daae58e-c060-4349-ba28-cfdeb5f6ee78'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.266859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '070571af27184314b2191fc2a3a1dd37', 'request_id': 'req-1daae58e-c060-4349-ba28-cfdeb5f6ee78'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.284632 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '070571af27184314b2191fc2a3a1dd37', 'request_id': 'req-1daae58e-c060-4349-ba28-cfdeb5f6ee78'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.311450 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561438f2-9a4b-449a-b236-3132753e7b94 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.323558 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] The token cache is being invalidate because project 070571af27184314b2191fc2a3a1dd37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:07.324990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.326551 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.331856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 902/1803] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:06 2022] DELETE /identity/v3/projects/070571af27184314b2191fc2a3a1dd37 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:07.336489 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.341727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.373080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:07.476561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76d876-6599-4ca0-8845-61ed4d46b285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.477720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76d876-6599-4ca0-8845-61ed4d46b285 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.478763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76d876-6599-4ca0-8845-61ed4d46b285 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.479511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76d876-6599-4ca0-8845-61ed4d46b285 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.480119 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76d876-6599-4ca0-8845-61ed4d46b285 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.482275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76d876-6599-4ca0-8845-61ed4d46b285 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.565223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 902/1804] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/projects => generated 371 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:07.566847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76d876-6599-4ca0-8845-61ed4d46b285 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.569222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-fd76d876-6599-4ca0-8845-61ed4d46b285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.574545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daae58e-c060-4349-ba28-cfdeb5f6ee78 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.596110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:07.679275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.680838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.680838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.681351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] PATH_INFO: `/v3/users/b36547d7dc1a489bb2268bbdd6d9c65b` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.683425 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b36547d7dc1a489bb2268bbdd6d9c65b)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.687053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.696096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76d876-6599-4ca0-8845-61ed4d46b285 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.703387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444581c6-a9af-43e1-abe3-d3bec761a524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.704402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444581c6-a9af-43e1-abe3-d3bec761a524 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.704830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444581c6-a9af-43e1-abe3-d3bec761a524 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.705301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444581c6-a9af-43e1-abe3-d3bec761a524 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.706233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444581c6-a9af-43e1-abe3-d3bec761a524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.710008 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444581c6-a9af-43e1-abe3-d3bec761a524 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.744419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 903/1805] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:07.750083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444581c6-a9af-43e1-abe3-d3bec761a524 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.763553 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444581c6-a9af-43e1-abe3-d3bec761a524 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.769761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f37da6-a718-4623-810b-9262d2a8f42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.770806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f37da6-a718-4623-810b-9262d2a8f42d admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.771382 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f37da6-a718-4623-810b-9262d2a8f42d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.771757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f37da6-a718-4623-810b-9262d2a8f42d admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.772560 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f37da6-a718-4623-810b-9262d2a8f42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.774175 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b36547d7dc1a489bb2268bbdd6d9c65b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.774842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f37da6-a718-4623-810b-9262d2a8f42d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.785050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b36547d7dc1a489bb2268bbdd6d9c65b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.786519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b36547d7dc1a489bb2268bbdd6d9c65b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.793008 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 904/1806] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:07.797974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f37da6-a718-4623-810b-9262d2a8f42d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.800132 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-39f37da6-a718-4623-810b-9262d2a8f42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.810343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b36547d7dc1a489bb2268bbdd6d9c65b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.812032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b36547d7dc1a489bb2268bbdd6d9c65b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.817430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f37da6-a718-4623-810b-9262d2a8f42d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.822909 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5907-217d-48fc-95f8-6e891eec611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.823626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5907-217d-48fc-95f8-6e891eec611f admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.823914 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5907-217d-48fc-95f8-6e891eec611f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.824204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5907-217d-48fc-95f8-6e891eec611f admin admin] PATH_INFO: `/v3/projects/2dbcab12e08e4efe813c5437e797bae6/users/5355eaae2b894eab8c6f696a8a1b3982/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.834088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 903/1807] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:07 2022] DELETE /identity/v3/users/b36547d7dc1a489bb2268bbdd6d9c65b => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:07.845215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5907-217d-48fc-95f8-6e891eec611f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dbcab12e08e4efe813c5437e797bae6, user_id=5355eaae2b894eab8c6f696a8a1b3982, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.848059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5907-217d-48fc-95f8-6e891eec611f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.870784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 905/1808] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:07 2022] PUT /identity/v3/projects/2dbcab12e08e4efe813c5437e797bae6/users/5355eaae2b894eab8c6f696a8a1b3982/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:07.875019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.930065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:07.977279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf89f5a-f5d6-4f98-8126-5820ff86683e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.088166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea75831-3f88-44a6-855d-ce9312a2e126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.089209 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea75831-3f88-44a6-855d-ce9312a2e126 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.089696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea75831-3f88-44a6-855d-ce9312a2e126 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.090170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea75831-3f88-44a6-855d-ce9312a2e126 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.091359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea75831-3f88-44a6-855d-ce9312a2e126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.094502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea75831-3f88-44a6-855d-ce9312a2e126 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.112234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 904/1809] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:08.113321 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5907-217d-48fc-95f8-6e891eec611f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.135508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5907-217d-48fc-95f8-6e891eec611f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.142155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91629988-ccef-4a0c-96b4-3f4dccc7660b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.143284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91629988-ccef-4a0c-96b4-3f4dccc7660b admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.143751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91629988-ccef-4a0c-96b4-3f4dccc7660b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.144038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91629988-ccef-4a0c-96b4-3f4dccc7660b admin admin] PATH_INFO: `/v3/projects/2dbcab12e08e4efe813c5437e797bae6/users/5355eaae2b894eab8c6f696a8a1b3982/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.147039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91629988-ccef-4a0c-96b4-3f4dccc7660b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dbcab12e08e4efe813c5437e797bae6, user_id=5355eaae2b894eab8c6f696a8a1b3982, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.150894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91629988-ccef-4a0c-96b4-3f4dccc7660b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.169088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 906/1810] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:08 2022] PUT /identity/v3/projects/2dbcab12e08e4efe813c5437e797bae6/users/5355eaae2b894eab8c6f696a8a1b3982/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:08.183959 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccd8c0-6ac1-45d7-8d6e-309944a4bd84 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.184439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccd8c0-6ac1-45d7-8d6e-309944a4bd84 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.184877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccd8c0-6ac1-45d7-8d6e-309944a4bd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.217915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f0ccd8c0-6ac1-45d7-8d6e-309944a4bd84 None None] MFA Rules not processed for user `5355eaae2b894eab8c6f696a8a1b3982`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:08.219173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f0ccd8c0-6ac1-45d7-8d6e-309944a4bd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:08.253486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91629988-ccef-4a0c-96b4-3f4dccc7660b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.257896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-91629988-ccef-4a0c-96b4-3f4dccc7660b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:08.377576 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91629988-ccef-4a0c-96b4-3f4dccc7660b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.385251 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 905/1811] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:08.388149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.389261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.389616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.389959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5 admin admin] PATH_INFO: `/v3/projects/95f67939e55745d087a2f1fb21bf7f03` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.392607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f67939e55745d087a2f1fb21bf7f03)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.397581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.401517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccd8c0-6ac1-45d7-8d6e-309944a4bd84 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.404292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f0ccd8c0-6ac1-45d7-8d6e-309944a4bd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:08.507430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f67939e55745d087a2f1fb21bf7f03', 'request_id': 'req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:08.518772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f67939e55745d087a2f1fb21bf7f03', 'request_id': 'req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:08.523576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccd8c0-6ac1-45d7-8d6e-309944a4bd84 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.528394 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:08.551948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f67939e55745d087a2f1fb21bf7f03', 'request_id': 'req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:08.553786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:08.575005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5 admin admin] The token cache is being invalidate because project 95f67939e55745d087a2f1fb21bf7f03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:08.575561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:08.576646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9f55a40a-1c2c-4e4d-8d5f-778e4f0a39e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:08.578516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 907/1812] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:08 2022] DELETE /identity/v3/projects/95f67939e55745d087a2f1fb21bf7f03 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:08.586702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f09582-5d09-420d-97a0-228eabacf9a6 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.587342 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f09582-5d09-420d-97a0-228eabacf9a6 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.587920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f09582-5d09-420d-97a0-228eabacf9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.613203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278b77d-4a72-4036-9445-6782c84186ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.614876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278b77d-4a72-4036-9445-6782c84186ac service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.615198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278b77d-4a72-4036-9445-6782c84186ac service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.615470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278b77d-4a72-4036-9445-6782c84186ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.617538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d278b77d-4a72-4036-9445-6782c84186ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:08.623269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b2f09582-5d09-420d-97a0-228eabacf9a6 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:08.626330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b2f09582-5d09-420d-97a0-228eabacf9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:08.736358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278b77d-4a72-4036-9445-6782c84186ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.738062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278b77d-4a72-4036-9445-6782c84186ac service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.772990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 906/1813] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:08.780982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 908/1814] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:08.796896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e7844-6efe-4ad2-b8b1-23f40124bc68 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.797668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e7844-6efe-4ad2-b8b1-23f40124bc68 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.800696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e7844-6efe-4ad2-b8b1-23f40124bc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.834045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-c78e7844-6efe-4ad2-b8b1-23f40124bc68 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:08.836186 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c78e7844-6efe-4ad2-b8b1-23f40124bc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:08.857070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 907/1815] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:08.859169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f09582-5d09-420d-97a0-228eabacf9a6 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.877314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f09582-5d09-420d-97a0-228eabacf9a6 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.885747 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408b77a-1a6a-49e3-b9aa-e463dd4eaea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.886587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408b77a-1a6a-49e3-b9aa-e463dd4eaea5 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.887038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408b77a-1a6a-49e3-b9aa-e463dd4eaea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.887593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408b77a-1a6a-49e3-b9aa-e463dd4eaea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.889067 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408b77a-1a6a-49e3-b9aa-e463dd4eaea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:08.889485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408b77a-1a6a-49e3-b9aa-e463dd4eaea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.892566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408b77a-1a6a-49e3-b9aa-e463dd4eaea5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.915650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 909/1816] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:08.922976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e7844-6efe-4ad2-b8b1-23f40124bc68 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.939864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e7844-6efe-4ad2-b8b1-23f40124bc68 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.948130 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cc92f-0fbb-4c3a-8267-ac0713921b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.948995 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc92f-0fbb-4c3a-8267-ac0713921b0f admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.949359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc92f-0fbb-4c3a-8267-ac0713921b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.949820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc92f-0fbb-4c3a-8267-ac0713921b0f admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.950825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cc92f-0fbb-4c3a-8267-ac0713921b0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.954198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cc92f-0fbb-4c3a-8267-ac0713921b0f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:09.004399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 908/1817] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:08 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:09.009818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408b77a-1a6a-49e3-b9aa-e463dd4eaea5 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:09.026132 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408b77a-1a6a-49e3-b9aa-e463dd4eaea5 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:09.151027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b99fb-77c9-48e9-aa46-e84a0f4c437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:09.152252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b99fb-77c9-48e9-aa46-e84a0f4c437a admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:09.152742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b99fb-77c9-48e9-aa46-e84a0f4c437a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:09.153181 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b99fb-77c9-48e9-aa46-e84a0f4c437a admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:09.154311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b99fb-77c9-48e9-aa46-e84a0f4c437a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:09.157938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b99fb-77c9-48e9-aa46-e84a0f4c437a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:09.194872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 910/1818] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:09 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:09.200720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cc92f-0fbb-4c3a-8267-ac0713921b0f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:09.212313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cc92f-0fbb-4c3a-8267-ac0713921b0f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:09.218411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8fdc6-2f33-46a0-8c99-b8922c428caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:09.219539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8fdc6-2f33-46a0-8c99-b8922c428caa admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:09.219991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8fdc6-2f33-46a0-8c99-b8922c428caa admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:09.220433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8fdc6-2f33-46a0-8c99-b8922c428caa admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:09.221610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8fdc6-2f33-46a0-8c99-b8922c428caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:09.224955 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8fdc6-2f33-46a0-8c99-b8922c428caa admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:09.241963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 909/1819] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:09.248574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b99fb-77c9-48e9-aa46-e84a0f4c437a admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:09.264958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b99fb-77c9-48e9-aa46-e84a0f4c437a admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:09.274203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235523ea-c4a5-436a-9476-d470f33b915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:09.275314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235523ea-c4a5-436a-9476-d470f33b915d admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:09.275801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235523ea-c4a5-436a-9476-d470f33b915d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:09.276275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235523ea-c4a5-436a-9476-d470f33b915d admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:09.277507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235523ea-c4a5-436a-9476-d470f33b915d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:09.280701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235523ea-c4a5-436a-9476-d470f33b915d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:09.304561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 911/1820] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:09.309377 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8fdc6-2f33-46a0-8c99-b8922c428caa admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:09.324565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8fdc6-2f33-46a0-8c99-b8922c428caa admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:09.334032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94113a-22d3-446c-ba45-dace26eed9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:09.335574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94113a-22d3-446c-ba45-dace26eed9d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:09.337293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94113a-22d3-446c-ba45-dace26eed9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:09.337895 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94113a-22d3-446c-ba45-dace26eed9d6 admin admin] PATH_INFO: `/v3/projects/5a94d540a2ce4857932ec8a6617b05ba/users/7a66d48fec444142a64d725524cfd4eb/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:09.371729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94113a-22d3-446c-ba45-dace26eed9d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a94d540a2ce4857932ec8a6617b05ba, user_id=7a66d48fec444142a64d725524cfd4eb, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:09.375203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94113a-22d3-446c-ba45-dace26eed9d6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:09.390087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235523ea-c4a5-436a-9476-d470f33b915d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:09.392208 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-235523ea-c4a5-436a-9476-d470f33b915d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:09.398022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 910/1821] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:09 2022] PUT /identity/v3/projects/5a94d540a2ce4857932ec8a6617b05ba/users/7a66d48fec444142a64d725524cfd4eb/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:09.411885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d524cc-7efc-4df4-8dde-540031725ddc None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:09.412252 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d524cc-7efc-4df4-8dde-540031725ddc None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:09.412702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d524cc-7efc-4df4-8dde-540031725ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:09.413465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235523ea-c4a5-436a-9476-d470f33b915d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:09.460565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-52d524cc-7efc-4df4-8dde-540031725ddc None None] MFA Rules not processed for user `7a66d48fec444142a64d725524cfd4eb`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:09.462104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-52d524cc-7efc-4df4-8dde-540031725ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:09.521599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:09.562928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40057bc9-97df-4b49-89f7-0f86b020e87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:09.564327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40057bc9-97df-4b49-89f7-0f86b020e87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:09.564919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40057bc9-97df-4b49-89f7-0f86b020e87c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:09.565618 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40057bc9-97df-4b49-89f7-0f86b020e87c admin admin] PATH_INFO: `/v3/users/7af77de115ae4d2fac7a3d2f52fe773b` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:09.568433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40057bc9-97df-4b49-89f7-0f86b020e87c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7af77de115ae4d2fac7a3d2f52fe773b)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:09.574484 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40057bc9-97df-4b49-89f7-0f86b020e87c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:09.713059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-40057bc9-97df-4b49-89f7-0f86b020e87c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7af77de115ae4d2fac7a3d2f52fe773b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:09.722397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:09.739345 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-40057bc9-97df-4b49-89f7-0f86b020e87c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7af77de115ae4d2fac7a3d2f52fe773b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:09.740974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-40057bc9-97df-4b49-89f7-0f86b020e87c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7af77de115ae4d2fac7a3d2f52fe773b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:09.742294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-40057bc9-97df-4b49-89f7-0f86b020e87c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7af77de115ae4d2fac7a3d2f52fe773b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:09.766702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-40057bc9-97df-4b49-89f7-0f86b020e87c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7af77de115ae4d2fac7a3d2f52fe773b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:09.777983 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 911/1822] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:09.790140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d524cc-7efc-4df4-8dde-540031725ddc None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:09.795622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-52d524cc-7efc-4df4-8dde-540031725ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:09.835496 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 912/1823] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:09 2022] DELETE /identity/v3/users/7af77de115ae4d2fac7a3d2f52fe773b => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:09.845105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b91226-2240-4644-bb71-83ef42359f99 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:09.845452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b91226-2240-4644-bb71-83ef42359f99 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:09.853986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b91226-2240-4644-bb71-83ef42359f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:09.965393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b0b91226-2240-4644-bb71-83ef42359f99 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:09.969575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d524cc-7efc-4df4-8dde-540031725ddc None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:09.980079 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b0b91226-2240-4644-bb71-83ef42359f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:10.130578 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88a3eb-b57b-4aa1-bfa7-93b74aada142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:10.132518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88a3eb-b57b-4aa1-bfa7-93b74aada142 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:10.133035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88a3eb-b57b-4aa1-bfa7-93b74aada142 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:10.133480 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88a3eb-b57b-4aa1-bfa7-93b74aada142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:10.136969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7b88a3eb-b57b-4aa1-bfa7-93b74aada142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:10.224018 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 913/1824] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:10.236981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b91226-2240-4644-bb71-83ef42359f99 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:10.238655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b0b91226-2240-4644-bb71-83ef42359f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:10.346067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88a3eb-b57b-4aa1-bfa7-93b74aada142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:10.347760 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88a3eb-b57b-4aa1-bfa7-93b74aada142 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:10.360994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 912/1825] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:10.406405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b91226-2240-4644-bb71-83ef42359f99 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:10.413372 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff45128-c89a-4b00-9936-f1e21a5e7847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:10.424087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff45128-c89a-4b00-9936-f1e21a5e7847 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:10.424631 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff45128-c89a-4b00-9936-f1e21a5e7847 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:10.425825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff45128-c89a-4b00-9936-f1e21a5e7847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:10.445683 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff45128-c89a-4b00-9936-f1e21a5e7847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:10.448174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff45128-c89a-4b00-9936-f1e21a5e7847 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:10.463276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 914/1826] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:12.674337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88a3eb-b57b-4aa1-bfa7-93b74aada142 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:12.685758 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7b88a3eb-b57b-4aa1-bfa7-93b74aada142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:12.757125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88a3eb-b57b-4aa1-bfa7-93b74aada142 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:12.763219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:12.764860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:12.765157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:12.765437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] PATH_INFO: `/v3/projects/0a987d86c86540bda66ee443f0b9a5af` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:12.766941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a987d86c86540bda66ee443f0b9a5af)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:12.768978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:12.887055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a987d86c86540bda66ee443f0b9a5af', 'request_id': 'req-fe85e905-485b-4742-a758-c58eca62057e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:12.897918 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a987d86c86540bda66ee443f0b9a5af', 'request_id': 'req-fe85e905-485b-4742-a758-c58eca62057e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:12.906913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:12.908972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a987d86c86540bda66ee443f0b9a5af', 'request_id': 'req-fe85e905-485b-4742-a758-c58eca62057e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:12.941253 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] The token cache is being invalidate because project 0a987d86c86540bda66ee443f0b9a5af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:12.943730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:12.945682 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:12.948354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 913/1827] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:12 2022] DELETE /identity/v3/projects/0a987d86c86540bda66ee443f0b9a5af => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:12.954918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff45128-c89a-4b00-9936-f1e21a5e7847 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:12.988380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff45128-c89a-4b00-9936-f1e21a5e7847 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:13.007900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:13.149905 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e93258-e9cd-4b03-9a3a-faeb4bb1078e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:13.151789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e93258-e9cd-4b03-9a3a-faeb4bb1078e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:13.152480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e93258-e9cd-4b03-9a3a-faeb4bb1078e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:13.153187 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e93258-e9cd-4b03-9a3a-faeb4bb1078e admin admin] PATH_INFO: `/v3/users/5355eaae2b894eab8c6f696a8a1b3982` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:13.156372 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e93258-e9cd-4b03-9a3a-faeb4bb1078e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5355eaae2b894eab8c6f696a8a1b3982)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:13.162202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e93258-e9cd-4b03-9a3a-faeb4bb1078e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:13.278692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-20e93258-e9cd-4b03-9a3a-faeb4bb1078e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5355eaae2b894eab8c6f696a8a1b3982'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.302705 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-20e93258-e9cd-4b03-9a3a-faeb4bb1078e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5355eaae2b894eab8c6f696a8a1b3982'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.304212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-20e93258-e9cd-4b03-9a3a-faeb4bb1078e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5355eaae2b894eab8c6f696a8a1b3982'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.304885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-20e93258-e9cd-4b03-9a3a-faeb4bb1078e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5355eaae2b894eab8c6f696a8a1b3982'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.322678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-20e93258-e9cd-4b03-9a3a-faeb4bb1078e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5355eaae2b894eab8c6f696a8a1b3982'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.368755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 915/1828] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:12 2022] DELETE /identity/v3/users/5355eaae2b894eab8c6f696a8a1b3982 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:14.024845 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.026537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:14.056081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85e905-485b-4742-a758-c58eca62057e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.154952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b71974-f430-4d84-a8a9-906710362295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.156069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b71974-f430-4d84-a8a9-906710362295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.156553 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b71974-f430-4d84-a8a9-906710362295 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.157023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b71974-f430-4d84-a8a9-906710362295 admin admin] PATH_INFO: `/v3/projects/2dbcab12e08e4efe813c5437e797bae6` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.159144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b71974-f430-4d84-a8a9-906710362295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dbcab12e08e4efe813c5437e797bae6)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.163758 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b71974-f430-4d84-a8a9-906710362295 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.241085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-05b71974-f430-4d84-a8a9-906710362295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dbcab12e08e4efe813c5437e797bae6', 'request_id': 'req-05b71974-f430-4d84-a8a9-906710362295'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:14.249533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-05b71974-f430-4d84-a8a9-906710362295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dbcab12e08e4efe813c5437e797bae6', 'request_id': 'req-05b71974-f430-4d84-a8a9-906710362295'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:14.257887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-05b71974-f430-4d84-a8a9-906710362295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dbcab12e08e4efe813c5437e797bae6', 'request_id': 'req-05b71974-f430-4d84-a8a9-906710362295'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:14.272254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-05b71974-f430-4d84-a8a9-906710362295 admin admin] The token cache is being invalidate because project 2dbcab12e08e4efe813c5437e797bae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:14.272540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-05b71974-f430-4d84-a8a9-906710362295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:14.273268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-05b71974-f430-4d84-a8a9-906710362295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:14.274822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 914/1829] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:14 2022] DELETE /identity/v3/projects/2dbcab12e08e4efe813c5437e797bae6 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:14.280577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f66e6-c630-4d0c-bbfd-1f244a9e85d7 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.281211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f66e6-c630-4d0c-bbfd-1f244a9e85d7 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.281211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f66e6-c630-4d0c-bbfd-1f244a9e85d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.304379 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ba6f66e6-c630-4d0c-bbfd-1f244a9e85d7 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:14.305638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ba6f66e6-c630-4d0c-bbfd-1f244a9e85d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:14.381175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 916/1830] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:14.389969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c3afa-66e6-4334-aecc-602326db03f6 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.390368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c3afa-66e6-4334-aecc-602326db03f6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.390638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c3afa-66e6-4334-aecc-602326db03f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.413883 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-dc8c3afa-66e6-4334-aecc-602326db03f6 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:14.415133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-dc8c3afa-66e6-4334-aecc-602326db03f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:14.424281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 915/1831] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:14.429539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f66e6-c630-4d0c-bbfd-1f244a9e85d7 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.441339 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f66e6-c630-4d0c-bbfd-1f244a9e85d7 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.448477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe394f9d-9ff0-48ca-8364-9cbd53289272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.449290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe394f9d-9ff0-48ca-8364-9cbd53289272 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.449660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe394f9d-9ff0-48ca-8364-9cbd53289272 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.450039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe394f9d-9ff0-48ca-8364-9cbd53289272 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.450851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe394f9d-9ff0-48ca-8364-9cbd53289272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:14.451338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe394f9d-9ff0-48ca-8364-9cbd53289272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.453239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe394f9d-9ff0-48ca-8364-9cbd53289272 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.468549 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 917/1832] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:14.473734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c3afa-66e6-4334-aecc-602326db03f6 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.484659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c3afa-66e6-4334-aecc-602326db03f6 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.491335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a411bd-282d-4a62-a46b-f5b8ce2ed316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.492024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a411bd-282d-4a62-a46b-f5b8ce2ed316 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.492299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a411bd-282d-4a62-a46b-f5b8ce2ed316 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.492572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a411bd-282d-4a62-a46b-f5b8ce2ed316 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.493225 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a411bd-282d-4a62-a46b-f5b8ce2ed316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.495226 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a411bd-282d-4a62-a46b-f5b8ce2ed316 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.529386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 916/1833] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:14.533509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe394f9d-9ff0-48ca-8364-9cbd53289272 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.553387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe394f9d-9ff0-48ca-8364-9cbd53289272 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.585310 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:14.657233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319adcd-e156-4282-8ede-384e1c263bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.658058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319adcd-e156-4282-8ede-384e1c263bae admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.658561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319adcd-e156-4282-8ede-384e1c263bae admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.658966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319adcd-e156-4282-8ede-384e1c263bae admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.659778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319adcd-e156-4282-8ede-384e1c263bae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.662839 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319adcd-e156-4282-8ede-384e1c263bae admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.687743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 918/1834] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:14.693407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a411bd-282d-4a62-a46b-f5b8ce2ed316 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.707333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a411bd-282d-4a62-a46b-f5b8ce2ed316 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.712897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02877a-dcd1-4af0-9f72-c87912eec088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.713521 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02877a-dcd1-4af0-9f72-c87912eec088 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.713792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02877a-dcd1-4af0-9f72-c87912eec088 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.714070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02877a-dcd1-4af0-9f72-c87912eec088 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.714711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02877a-dcd1-4af0-9f72-c87912eec088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.716376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02877a-dcd1-4af0-9f72-c87912eec088 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.727846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 917/1835] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:14.733500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319adcd-e156-4282-8ede-384e1c263bae admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.743966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319adcd-e156-4282-8ede-384e1c263bae admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.748920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0442bdc-90be-4294-8a8c-1505b8d9030d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.749541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0442bdc-90be-4294-8a8c-1505b8d9030d admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.749815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0442bdc-90be-4294-8a8c-1505b8d9030d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.750084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0442bdc-90be-4294-8a8c-1505b8d9030d admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.750740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0442bdc-90be-4294-8a8c-1505b8d9030d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.752414 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0442bdc-90be-4294-8a8c-1505b8d9030d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.769675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 919/1836] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:14.780549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02877a-dcd1-4af0-9f72-c87912eec088 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.804181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02877a-dcd1-4af0-9f72-c87912eec088 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.812304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f286c-da83-4b63-805a-9be7e3935258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.813684 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f286c-da83-4b63-805a-9be7e3935258 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.814301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f286c-da83-4b63-805a-9be7e3935258 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.815171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f286c-da83-4b63-805a-9be7e3935258 admin admin] PATH_INFO: `/v3/projects/7700613a4a4c447a830e3541b12b77e6/users/ac756f53a5114329b649c14fdfbbad02/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.842424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f286c-da83-4b63-805a-9be7e3935258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7700613a4a4c447a830e3541b12b77e6, user_id=ac756f53a5114329b649c14fdfbbad02, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.849309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f286c-da83-4b63-805a-9be7e3935258 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.870380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 918/1837] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:14 2022] PUT /identity/v3/projects/7700613a4a4c447a830e3541b12b77e6/users/ac756f53a5114329b649c14fdfbbad02/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:14.884536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464c735-60c9-4e4b-8f43-57703a25a5fc None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.884748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464c735-60c9-4e4b-8f43-57703a25a5fc None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.885017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464c735-60c9-4e4b-8f43-57703a25a5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.915209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0464c735-60c9-4e4b-8f43-57703a25a5fc None None] MFA Rules not processed for user `ac756f53a5114329b649c14fdfbbad02`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:14.916587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0464c735-60c9-4e4b-8f43-57703a25a5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:15.003873 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 920/1838] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:15.009486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f286c-da83-4b63-805a-9be7e3935258 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.031675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:15.113380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f286c-da83-4b63-805a-9be7e3935258 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.121731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203953aa-c8a4-461f-93ff-2c8c9d5dc11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.122695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203953aa-c8a4-461f-93ff-2c8c9d5dc11d admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.123176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203953aa-c8a4-461f-93ff-2c8c9d5dc11d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.124450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203953aa-c8a4-461f-93ff-2c8c9d5dc11d admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.125014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203953aa-c8a4-461f-93ff-2c8c9d5dc11d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.129189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203953aa-c8a4-461f-93ff-2c8c9d5dc11d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.194461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 919/1839] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:15 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:15.198647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464c735-60c9-4e4b-8f43-57703a25a5fc None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.211745 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464c735-60c9-4e4b-8f43-57703a25a5fc None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.282526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294575c-a571-4eaf-99a1-03ed0b00085b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.283338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294575c-a571-4eaf-99a1-03ed0b00085b admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.283475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294575c-a571-4eaf-99a1-03ed0b00085b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.283711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294575c-a571-4eaf-99a1-03ed0b00085b admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.284362 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294575c-a571-4eaf-99a1-03ed0b00085b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.286992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294575c-a571-4eaf-99a1-03ed0b00085b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.312314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 921/1840] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:15 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:15.318858 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203953aa-c8a4-461f-93ff-2c8c9d5dc11d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.337013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203953aa-c8a4-461f-93ff-2c8c9d5dc11d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.345200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a8b22-c3c1-481c-9196-deda7226e1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.346010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a8b22-c3c1-481c-9196-deda7226e1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.346371 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a8b22-c3c1-481c-9196-deda7226e1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.346703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a8b22-c3c1-481c-9196-deda7226e1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.348466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a8b22-c3c1-481c-9196-deda7226e1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.351339 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a8b22-c3c1-481c-9196-deda7226e1d6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.372118 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 920/1841] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:15.376143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294575c-a571-4eaf-99a1-03ed0b00085b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.387240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294575c-a571-4eaf-99a1-03ed0b00085b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.392443 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6c9c1-a8b8-4125-a75a-201e16351131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.393114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6c9c1-a8b8-4125-a75a-201e16351131 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.393432 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6c9c1-a8b8-4125-a75a-201e16351131 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.393776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6c9c1-a8b8-4125-a75a-201e16351131 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.394526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6c9c1-a8b8-4125-a75a-201e16351131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.396426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6c9c1-a8b8-4125-a75a-201e16351131 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.408896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 922/1842] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:15.415476 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a8b22-c3c1-481c-9196-deda7226e1d6 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.426944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a8b22-c3c1-481c-9196-deda7226e1d6 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.431934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313e815-4a84-4348-89cb-ffc905bebd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.432637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313e815-4a84-4348-89cb-ffc905bebd2f admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.432913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313e815-4a84-4348-89cb-ffc905bebd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.433183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313e815-4a84-4348-89cb-ffc905bebd2f admin admin] PATH_INFO: `/v3/projects/63ab3819bdaa497ba6e65f03a701c636/users/75e906a711e148caa28a12a7dc4b9327/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.450546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313e815-4a84-4348-89cb-ffc905bebd2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=63ab3819bdaa497ba6e65f03a701c636, user_id=75e906a711e148caa28a12a7dc4b9327, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.453824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313e815-4a84-4348-89cb-ffc905bebd2f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.468438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 921/1843] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:15 2022] PUT /identity/v3/projects/63ab3819bdaa497ba6e65f03a701c636/users/75e906a711e148caa28a12a7dc4b9327/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:15.480437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077f51c-9241-4bc7-aa85-c8f9c9259be8 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.480723 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077f51c-9241-4bc7-aa85-c8f9c9259be8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.481083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077f51c-9241-4bc7-aa85-c8f9c9259be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.496400 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:15.514577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-6077f51c-9241-4bc7-aa85-c8f9c9259be8 None None] MFA Rules not processed for user `75e906a711e148caa28a12a7dc4b9327`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:15.516161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6077f51c-9241-4bc7-aa85-c8f9c9259be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:15.596555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 923/1844] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:15.602514 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313e815-4a84-4348-89cb-ffc905bebd2f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.732917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313e815-4a84-4348-89cb-ffc905bebd2f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.749634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367ce64-f43b-4e01-adef-1d04181a0394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.750371 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367ce64-f43b-4e01-adef-1d04181a0394 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.750866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367ce64-f43b-4e01-adef-1d04181a0394 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.751053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367ce64-f43b-4e01-adef-1d04181a0394 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.751767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367ce64-f43b-4e01-adef-1d04181a0394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.754636 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367ce64-f43b-4e01-adef-1d04181a0394 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.799331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 922/1845] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:15 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:15.802886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077f51c-9241-4bc7-aa85-c8f9c9259be8 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.870217 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077f51c-9241-4bc7-aa85-c8f9c9259be8 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:16.121407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cc34f-c59a-41ee-a570-255a4689de0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:16.122045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cc34f-c59a-41ee-a570-255a4689de0b admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:16.122326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cc34f-c59a-41ee-a570-255a4689de0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:16.122628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cc34f-c59a-41ee-a570-255a4689de0b admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:16.123392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8cc34f-c59a-41ee-a570-255a4689de0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:16.125434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8cc34f-c59a-41ee-a570-255a4689de0b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:16.153716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 924/1846] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:15 2022] POST /identity/v3/users => generated 299 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:16.161692 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367ce64-f43b-4e01-adef-1d04181a0394 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:16.197157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367ce64-f43b-4e01-adef-1d04181a0394 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:16.220349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ab32b-8dfb-4aa1-a705-7aa3b5f23f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:16.220998 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ab32b-8dfb-4aa1-a705-7aa3b5f23f30 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:16.221282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ab32b-8dfb-4aa1-a705-7aa3b5f23f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:16.221681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ab32b-8dfb-4aa1-a705-7aa3b5f23f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:16.222735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ab32b-8dfb-4aa1-a705-7aa3b5f23f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:16.228897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ab32b-8dfb-4aa1-a705-7aa3b5f23f30 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:16.281304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 923/1847] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:16.286780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cc34f-c59a-41ee-a570-255a4689de0b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:16.306535 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cc34f-c59a-41ee-a570-255a4689de0b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:16.316869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96bd50-d334-46e3-99c9-c059e93dca95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:16.318413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96bd50-d334-46e3-99c9-c059e93dca95 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:16.318865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96bd50-d334-46e3-99c9-c059e93dca95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:16.319432 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96bd50-d334-46e3-99c9-c059e93dca95 admin admin] PATH_INFO: `/v3/projects/032e879fb7e24676973d6a08c44b6d58/users/4785cd91963f4be8b9462f530d3c272b/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:16.356034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96bd50-d334-46e3-99c9-c059e93dca95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032e879fb7e24676973d6a08c44b6d58, user_id=4785cd91963f4be8b9462f530d3c272b, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:16.362309 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96bd50-d334-46e3-99c9-c059e93dca95 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:16.408138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 925/1848] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:16 2022] PUT /identity/v3/projects/032e879fb7e24676973d6a08c44b6d58/users/4785cd91963f4be8b9462f530d3c272b/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:16.413811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ab32b-8dfb-4aa1-a705-7aa3b5f23f30 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:16.483441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ab32b-8dfb-4aa1-a705-7aa3b5f23f30 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:16.608865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:16.653578 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec30e3-d5c0-4af3-8e0f-2f4a43bd9028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:16.655184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec30e3-d5c0-4af3-8e0f-2f4a43bd9028 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:16.655184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec30e3-d5c0-4af3-8e0f-2f4a43bd9028 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:16.655184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec30e3-d5c0-4af3-8e0f-2f4a43bd9028 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:16.655937 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec30e3-d5c0-4af3-8e0f-2f4a43bd9028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:16.659272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec30e3-d5c0-4af3-8e0f-2f4a43bd9028 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:16.691051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 924/1849] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:16.694504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96bd50-d334-46e3-99c9-c059e93dca95 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:16.708999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96bd50-d334-46e3-99c9-c059e93dca95 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:16.725144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a33fa5-a042-4496-84f0-10bf922ec204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:16.725964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a33fa5-a042-4496-84f0-10bf922ec204 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:16.726287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a33fa5-a042-4496-84f0-10bf922ec204 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:16.726585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a33fa5-a042-4496-84f0-10bf922ec204 admin admin] PATH_INFO: `/v3/projects/032e879fb7e24676973d6a08c44b6d58/users/4785cd91963f4be8b9462f530d3c272b/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:16.739207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a33fa5-a042-4496-84f0-10bf922ec204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032e879fb7e24676973d6a08c44b6d58, user_id=4785cd91963f4be8b9462f530d3c272b, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:16.742661 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a33fa5-a042-4496-84f0-10bf922ec204 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:16.757655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 926/1850] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:16 2022] PUT /identity/v3/projects/032e879fb7e24676973d6a08c44b6d58/users/4785cd91963f4be8b9462f530d3c272b/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:16.776928 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b49842-aed8-4c03-9066-d1967081ab98 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:16.777211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b49842-aed8-4c03-9066-d1967081ab98 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:16.778301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b49842-aed8-4c03-9066-d1967081ab98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:16.843622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-06b49842-aed8-4c03-9066-d1967081ab98 None None] MFA Rules not processed for user `4785cd91963f4be8b9462f530d3c272b`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:16.846615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-06b49842-aed8-4c03-9066-d1967081ab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:17.079992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 925/1851] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:17.115968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a33fa5-a042-4496-84f0-10bf922ec204 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:17.125401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-13a33fa5-a042-4496-84f0-10bf922ec204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:17.246903 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b49842-aed8-4c03-9066-d1967081ab98 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:17.249475 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-06b49842-aed8-4c03-9066-d1967081ab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:17.383899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:17.450426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a33fa5-a042-4496-84f0-10bf922ec204 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:17.481237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:17.484315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:17.484315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:17.484315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] PATH_INFO: `/v3/users/312f2b29f1ca41b7957bafb01326cf9e` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:17.486606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=312f2b29f1ca41b7957bafb01326cf9e)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:17.491899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:17.568643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b49842-aed8-4c03-9066-d1967081ab98 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:17.594693 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e10e7-42f4-49d7-895f-2640e2775e38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:17.599700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e10e7-42f4-49d7-895f-2640e2775e38 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:17.600495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e10e7-42f4-49d7-895f-2640e2775e38 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:17.603919 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e10e7-42f4-49d7-895f-2640e2775e38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:17.640612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e10e7-42f4-49d7-895f-2640e2775e38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:17.650437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e10e7-42f4-49d7-895f-2640e2775e38 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:17.691299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 926/1852] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:17.721022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '312f2b29f1ca41b7957bafb01326cf9e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.740826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '312f2b29f1ca41b7957bafb01326cf9e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.742600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '312f2b29f1ca41b7957bafb01326cf9e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.743762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '312f2b29f1ca41b7957bafb01326cf9e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.762743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '312f2b29f1ca41b7957bafb01326cf9e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.795371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 927/1853] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:17 2022] DELETE /identity/v3/users/312f2b29f1ca41b7957bafb01326cf9e => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:17.802910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c39f5c-ecd2-49b9-8058-5d6dac8107ec None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:17.803662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c39f5c-ecd2-49b9-8058-5d6dac8107ec None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:17.804334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c39f5c-ecd2-49b9-8058-5d6dac8107ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:17.907762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-c8c39f5c-ecd2-49b9-8058-5d6dac8107ec None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:17.924363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c8c39f5c-ecd2-49b9-8058-5d6dac8107ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:18.124307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 927/1854] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:18.150698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:18.151665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:18.335148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da193ec-da42-4aab-ad1e-163058cd9d93 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:18.340717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f706a-ffa3-44ba-b767-fb83fa4260cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:18.342563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f706a-ffa3-44ba-b767-fb83fa4260cc service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:18.343246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f706a-ffa3-44ba-b767-fb83fa4260cc service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:18.343674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f706a-ffa3-44ba-b767-fb83fa4260cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:18.356267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f706a-ffa3-44ba-b767-fb83fa4260cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:18.358324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f706a-ffa3-44ba-b767-fb83fa4260cc service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:18.376069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 928/1855] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:19.050113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c39f5c-ecd2-49b9-8058-5d6dac8107ec None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:19.054084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c8c39f5c-ecd2-49b9-8058-5d6dac8107ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:19.087945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:19.167276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c39f5c-ecd2-49b9-8058-5d6dac8107ec None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:19.174128 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259260f3-8ca9-4bbd-904d-c90d4ffa4691 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:19.175879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259260f3-8ca9-4bbd-904d-c90d4ffa4691 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:19.175879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259260f3-8ca9-4bbd-904d-c90d4ffa4691 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:19.177647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259260f3-8ca9-4bbd-904d-c90d4ffa4691 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:19.180963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-259260f3-8ca9-4bbd-904d-c90d4ffa4691 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:19.260009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259260f3-8ca9-4bbd-904d-c90d4ffa4691 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:19.261506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259260f3-8ca9-4bbd-904d-c90d4ffa4691 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:19.273692 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 928/1856] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:19.501089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f706a-ffa3-44ba-b767-fb83fa4260cc service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:19.504726 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-513f706a-ffa3-44ba-b767-fb83fa4260cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:19.556025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f706a-ffa3-44ba-b767-fb83fa4260cc service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:19.586870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:19.588474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:19.589202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:19.590950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] PATH_INFO: `/v3/projects/446294e7931b4032a7b7eb8422a74c38` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:19.595642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] RBAC: Authorizing `identity:delete_project(project_id=446294e7931b4032a7b7eb8422a74c38)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:19.600218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:19.714432 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '446294e7931b4032a7b7eb8422a74c38', 'request_id': 'req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.729083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '446294e7931b4032a7b7eb8422a74c38', 'request_id': 'req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.738911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '446294e7931b4032a7b7eb8422a74c38', 'request_id': 'req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.744673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:19.757256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] The token cache is being invalidate because project 446294e7931b4032a7b7eb8422a74c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:19.757991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.759504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.768764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259260f3-8ca9-4bbd-904d-c90d4ffa4691 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:19.769280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 929/1857] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:19 2022] DELETE /identity/v3/projects/446294e7931b4032a7b7eb8422a74c38 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:19.786669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259260f3-8ca9-4bbd-904d-c90d4ffa4691 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:19.922041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d0f3a-f970-4e09-9153-142b7eaf8b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:19.922996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d0f3a-f970-4e09-9153-142b7eaf8b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:19.924196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d0f3a-f970-4e09-9153-142b7eaf8b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:19.924196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d0f3a-f970-4e09-9153-142b7eaf8b23 admin admin] PATH_INFO: `/v3/users/f124f9478fed45feabbe1111bc4f622a` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:19.925796 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d0f3a-f970-4e09-9153-142b7eaf8b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f124f9478fed45feabbe1111bc4f622a)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:19.929263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d0f3a-f970-4e09-9153-142b7eaf8b23 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:19.997630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2a8d0f3a-f970-4e09-9153-142b7eaf8b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f124f9478fed45feabbe1111bc4f622a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.011001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2a8d0f3a-f970-4e09-9153-142b7eaf8b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f124f9478fed45feabbe1111bc4f622a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.012130 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2a8d0f3a-f970-4e09-9153-142b7eaf8b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f124f9478fed45feabbe1111bc4f622a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.021016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:20.030200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2a8d0f3a-f970-4e09-9153-142b7eaf8b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f124f9478fed45feabbe1111bc4f622a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.031183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2a8d0f3a-f970-4e09-9153-142b7eaf8b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f124f9478fed45feabbe1111bc4f622a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.051905 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 929/1858] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:19 2022] DELETE /identity/v3/users/f124f9478fed45feabbe1111bc4f622a => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:20.428886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.431471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:20.452840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cc5ee-4521-4390-b34f-44a76a6bdafe admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.564647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019c8b6-a7cc-46eb-854a-59574efe16e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.565686 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019c8b6-a7cc-46eb-854a-59574efe16e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.566028 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019c8b6-a7cc-46eb-854a-59574efe16e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.566370 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019c8b6-a7cc-46eb-854a-59574efe16e9 admin admin] PATH_INFO: `/v3/projects/2369dbc9c167413e99d81cbf889ae607` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.568899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019c8b6-a7cc-46eb-854a-59574efe16e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2369dbc9c167413e99d81cbf889ae607)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.572601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019c8b6-a7cc-46eb-854a-59574efe16e9 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:20.675782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a019c8b6-a7cc-46eb-854a-59574efe16e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2369dbc9c167413e99d81cbf889ae607', 'request_id': 'req-a019c8b6-a7cc-46eb-854a-59574efe16e9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.690273 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a019c8b6-a7cc-46eb-854a-59574efe16e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2369dbc9c167413e99d81cbf889ae607', 'request_id': 'req-a019c8b6-a7cc-46eb-854a-59574efe16e9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.701412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a019c8b6-a7cc-46eb-854a-59574efe16e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2369dbc9c167413e99d81cbf889ae607', 'request_id': 'req-a019c8b6-a7cc-46eb-854a-59574efe16e9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.725459 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a019c8b6-a7cc-46eb-854a-59574efe16e9 admin admin] The token cache is being invalidate because project 2369dbc9c167413e99d81cbf889ae607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:20.725985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a019c8b6-a7cc-46eb-854a-59574efe16e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.727168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a019c8b6-a7cc-46eb-854a-59574efe16e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.729891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 930/1859] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:20 2022] DELETE /identity/v3/projects/2369dbc9c167413e99d81cbf889ae607 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:20.735833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad2d47-888f-48f2-b220-f2175a8e928d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.736349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad2d47-888f-48f2-b220-f2175a8e928d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.736624 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad2d47-888f-48f2-b220-f2175a8e928d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.758809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b6ad2d47-888f-48f2-b220-f2175a8e928d None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:20.760151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b6ad2d47-888f-48f2-b220-f2175a8e928d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:20.825140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 930/1860] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:20.833674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594aaaf-522f-45c1-9b04-7b90945bbacc None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.833961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594aaaf-522f-45c1-9b04-7b90945bbacc None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.834256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594aaaf-522f-45c1-9b04-7b90945bbacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.856337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-c594aaaf-522f-45c1-9b04-7b90945bbacc None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:20.857592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c594aaaf-522f-45c1-9b04-7b90945bbacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:20.866477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 931/1861] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:20.871673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad2d47-888f-48f2-b220-f2175a8e928d None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.887382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad2d47-888f-48f2-b220-f2175a8e928d None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.905834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0cf04-77dd-489b-ac61-3d2a970206d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.907297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0cf04-77dd-489b-ac61-3d2a970206d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.908032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0cf04-77dd-489b-ac61-3d2a970206d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.908528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0cf04-77dd-489b-ac61-3d2a970206d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.909780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0cf04-77dd-489b-ac61-3d2a970206d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:20.910334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0cf04-77dd-489b-ac61-3d2a970206d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.914320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0cf04-77dd-489b-ac61-3d2a970206d9 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:20.937895 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 931/1862] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:20.944807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594aaaf-522f-45c1-9b04-7b90945bbacc None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.961655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594aaaf-522f-45c1-9b04-7b90945bbacc None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.969756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ce794-f3af-45b7-9b80-a999263a8d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.970795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ce794-f3af-45b7-9b80-a999263a8d09 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.971328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ce794-f3af-45b7-9b80-a999263a8d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.971758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ce794-f3af-45b7-9b80-a999263a8d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.973471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ce794-f3af-45b7-9b80-a999263a8d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.977007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ce794-f3af-45b7-9b80-a999263a8d09 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.025138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 932/1863] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:20 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:21.031749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0cf04-77dd-489b-ac61-3d2a970206d9 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.052496 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0cf04-77dd-489b-ac61-3d2a970206d9 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.152877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4f573-2782-4deb-94d0-3f09c01121b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.153511 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4f573-2782-4deb-94d0-3f09c01121b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.153779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4f573-2782-4deb-94d0-3f09c01121b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.154272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4f573-2782-4deb-94d0-3f09c01121b6 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.154690 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4f573-2782-4deb-94d0-3f09c01121b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.156790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4f573-2782-4deb-94d0-3f09c01121b6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.187836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 932/1864] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:21 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:21.192091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ce794-f3af-45b7-9b80-a999263a8d09 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.205785 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ce794-f3af-45b7-9b80-a999263a8d09 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.211621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446722b-654f-4b76-ae0e-cfa1fd044e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.212258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446722b-654f-4b76-ae0e-cfa1fd044e58 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.212540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446722b-654f-4b76-ae0e-cfa1fd044e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.212831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446722b-654f-4b76-ae0e-cfa1fd044e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.213481 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446722b-654f-4b76-ae0e-cfa1fd044e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.215180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446722b-654f-4b76-ae0e-cfa1fd044e58 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.229114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 933/1865] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:21.236662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4f573-2782-4deb-94d0-3f09c01121b6 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.249730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4f573-2782-4deb-94d0-3f09c01121b6 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.255620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbf7dd-b9d3-4ddb-adad-bb5142adbfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.256248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbf7dd-b9d3-4ddb-adad-bb5142adbfe4 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.256598 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbf7dd-b9d3-4ddb-adad-bb5142adbfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.256866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbf7dd-b9d3-4ddb-adad-bb5142adbfe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.257545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbf7dd-b9d3-4ddb-adad-bb5142adbfe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.259200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbf7dd-b9d3-4ddb-adad-bb5142adbfe4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.275602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 933/1866] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:21.280172 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446722b-654f-4b76-ae0e-cfa1fd044e58 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.291035 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446722b-654f-4b76-ae0e-cfa1fd044e58 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.295832 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09b3bd-e59c-4776-bc9a-e39a780a1464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.296513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09b3bd-e59c-4776-bc9a-e39a780a1464 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.296791 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09b3bd-e59c-4776-bc9a-e39a780a1464 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.297063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09b3bd-e59c-4776-bc9a-e39a780a1464 admin admin] PATH_INFO: `/v3/projects/1ddb7c87c39a4c148da5893b62f94ad3/users/364a0a76f6ac45deb80e1ef8257f962f/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.312967 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09b3bd-e59c-4776-bc9a-e39a780a1464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ddb7c87c39a4c148da5893b62f94ad3, user_id=364a0a76f6ac45deb80e1ef8257f962f, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.316239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09b3bd-e59c-4776-bc9a-e39a780a1464 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.332039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 934/1867] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:21 2022] PUT /identity/v3/projects/1ddb7c87c39a4c148da5893b62f94ad3/users/364a0a76f6ac45deb80e1ef8257f962f/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:21.344583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176368be-ea01-4a43-98b6-688f4d454200 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.344863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176368be-ea01-4a43-98b6-688f4d454200 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.345134 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176368be-ea01-4a43-98b6-688f4d454200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.376417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-176368be-ea01-4a43-98b6-688f4d454200 None None] MFA Rules not processed for user `364a0a76f6ac45deb80e1ef8257f962f`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:21.377649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-176368be-ea01-4a43-98b6-688f4d454200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:21.430540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:21.469799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 934/1868] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:21.474691 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09b3bd-e59c-4776-bc9a-e39a780a1464 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.506261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:21.560847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09b3bd-e59c-4776-bc9a-e39a780a1464 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.565824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b51a1d-945e-419e-9d41-dc6bea46b5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.567213 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b51a1d-945e-419e-9d41-dc6bea46b5be admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.567361 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b51a1d-945e-419e-9d41-dc6bea46b5be admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.567640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b51a1d-945e-419e-9d41-dc6bea46b5be admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.568290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b51a1d-945e-419e-9d41-dc6bea46b5be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.570247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b51a1d-945e-419e-9d41-dc6bea46b5be admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.603267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 935/1869] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:21 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:21.608143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176368be-ea01-4a43-98b6-688f4d454200 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.623914 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176368be-ea01-4a43-98b6-688f4d454200 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.722001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57125f38-38f3-4e2d-a20b-4608d9971008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.722624 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57125f38-38f3-4e2d-a20b-4608d9971008 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.724131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57125f38-38f3-4e2d-a20b-4608d9971008 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.724131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57125f38-38f3-4e2d-a20b-4608d9971008 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.724492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57125f38-38f3-4e2d-a20b-4608d9971008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.727005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57125f38-38f3-4e2d-a20b-4608d9971008 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.773218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 935/1870] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:21 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:21.774510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b51a1d-945e-419e-9d41-dc6bea46b5be admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.798354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b51a1d-945e-419e-9d41-dc6bea46b5be admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.813125 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ecaac-79b4-41ba-b1ca-87bfcd992d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.814357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ecaac-79b4-41ba-b1ca-87bfcd992d73 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.815188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ecaac-79b4-41ba-b1ca-87bfcd992d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.817151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ecaac-79b4-41ba-b1ca-87bfcd992d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.820245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ecaac-79b4-41ba-b1ca-87bfcd992d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.822540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ecaac-79b4-41ba-b1ca-87bfcd992d73 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.868620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 936/1871] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:21.874790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57125f38-38f3-4e2d-a20b-4608d9971008 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.906686 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57125f38-38f3-4e2d-a20b-4608d9971008 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.930587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae9ae6-450b-4df8-9223-84bb47c1cd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.931538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae9ae6-450b-4df8-9223-84bb47c1cd55 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.939263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae9ae6-450b-4df8-9223-84bb47c1cd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.939263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae9ae6-450b-4df8-9223-84bb47c1cd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.940580 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae9ae6-450b-4df8-9223-84bb47c1cd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.945496 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae9ae6-450b-4df8-9223-84bb47c1cd55 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.967619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 936/1872] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:21.974460 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ecaac-79b4-41ba-b1ca-87bfcd992d73 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.988096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ecaac-79b4-41ba-b1ca-87bfcd992d73 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.996902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b82e2e-d836-4802-b4c3-0d0df9e5697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.998705 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b82e2e-d836-4802-b4c3-0d0df9e5697b admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.999269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b82e2e-d836-4802-b4c3-0d0df9e5697b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.999493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b82e2e-d836-4802-b4c3-0d0df9e5697b admin admin] PATH_INFO: `/v3/projects/89c1bd62ecad464da702ab840d152b13/users/1656370fe93a42de8791f2df634ea363/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:22.029771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b82e2e-d836-4802-b4c3-0d0df9e5697b admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c1bd62ecad464da702ab840d152b13, user_id=1656370fe93a42de8791f2df634ea363, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:22.034630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b82e2e-d836-4802-b4c3-0d0df9e5697b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:22.094773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 937/1873] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:21 2022] PUT /identity/v3/projects/89c1bd62ecad464da702ab840d152b13/users/1656370fe93a42de8791f2df634ea363/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:22.110459 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe1877-8255-43c1-9077-f054deb0db9b None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:22.111050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe1877-8255-43c1-9077-f054deb0db9b None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:22.111420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe1877-8255-43c1-9077-f054deb0db9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:22.184985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4afe1877-8255-43c1-9077-f054deb0db9b None None] MFA Rules not processed for user `1656370fe93a42de8791f2df634ea363`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:22.187547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4afe1877-8255-43c1-9077-f054deb0db9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:22.324157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b82e2e-d836-4802-b4c3-0d0df9e5697b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:22.326054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d4b82e2e-d836-4802-b4c3-0d0df9e5697b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:22.376837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 937/1874] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:22.384106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe1877-8255-43c1-9077-f054deb0db9b None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:22.552937 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:22.594145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b82e2e-d836-4802-b4c3-0d0df9e5697b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:22.596195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe1877-8255-43c1-9077-f054deb0db9b None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:22.609661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd1968-97e3-4e18-ac2e-dc1e03f1e57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:22.610363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd1968-97e3-4e18-ac2e-dc1e03f1e57e admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:22.610645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd1968-97e3-4e18-ac2e-dc1e03f1e57e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:22.610973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd1968-97e3-4e18-ac2e-dc1e03f1e57e admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:22.611863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd1968-97e3-4e18-ac2e-dc1e03f1e57e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:22.614428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd1968-97e3-4e18-ac2e-dc1e03f1e57e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:22.623099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44190653-f28a-4a33-8d03-fe07fd13ea77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:22.626613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44190653-f28a-4a33-8d03-fe07fd13ea77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:22.628749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44190653-f28a-4a33-8d03-fe07fd13ea77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:22.631346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44190653-f28a-4a33-8d03-fe07fd13ea77 admin admin] PATH_INFO: `/v3/users/7a66d48fec444142a64d725524cfd4eb` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:22.635084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44190653-f28a-4a33-8d03-fe07fd13ea77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a66d48fec444142a64d725524cfd4eb)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:22.641712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44190653-f28a-4a33-8d03-fe07fd13ea77 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:22.691217 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 938/1875] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:22 2022] POST /identity/v3/projects => generated 377 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:22.696615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd1968-97e3-4e18-ac2e-dc1e03f1e57e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:22.810783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-44190653-f28a-4a33-8d03-fe07fd13ea77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a66d48fec444142a64d725524cfd4eb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:22.822567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd1968-97e3-4e18-ac2e-dc1e03f1e57e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:22.838503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c229273-1b87-47c0-822d-754f8d6b5d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:22.839507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c229273-1b87-47c0-822d-754f8d6b5d8f admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:22.840014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c229273-1b87-47c0-822d-754f8d6b5d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:22.840680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c229273-1b87-47c0-822d-754f8d6b5d8f admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:22.841626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c229273-1b87-47c0-822d-754f8d6b5d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:22.843964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c229273-1b87-47c0-822d-754f8d6b5d8f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:22.874829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-44190653-f28a-4a33-8d03-fe07fd13ea77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a66d48fec444142a64d725524cfd4eb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:22.878583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 939/1876] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:22 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:22.880180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-44190653-f28a-4a33-8d03-fe07fd13ea77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a66d48fec444142a64d725524cfd4eb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:22.883453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-44190653-f28a-4a33-8d03-fe07fd13ea77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a66d48fec444142a64d725524cfd4eb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:22.884533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c229273-1b87-47c0-822d-754f8d6b5d8f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:22.888626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-8c229273-1b87-47c0-822d-754f8d6b5d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:22.907105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-44190653-f28a-4a33-8d03-fe07fd13ea77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a66d48fec444142a64d725524cfd4eb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:22.924381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:22.927799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c229273-1b87-47c0-822d-754f8d6b5d8f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:22.939903 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e82fd7-d270-4830-9d34-faff021dd44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:22.940550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e82fd7-d270-4830-9d34-faff021dd44e admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:22.940882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e82fd7-d270-4830-9d34-faff021dd44e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:22.941242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e82fd7-d270-4830-9d34-faff021dd44e admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:22.941881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e82fd7-d270-4830-9d34-faff021dd44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:22.948174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e82fd7-d270-4830-9d34-faff021dd44e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:22.955257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 938/1877] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:22 2022] DELETE /identity/v3/users/7a66d48fec444142a64d725524cfd4eb => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:22.965071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a015b1c-b654-4f50-a2ac-59531bd096af None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:22.965634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a015b1c-b654-4f50-a2ac-59531bd096af None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:22.966118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a015b1c-b654-4f50-a2ac-59531bd096af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:22.990453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 940/1878] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:22.994958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e82fd7-d270-4830-9d34-faff021dd44e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:23.016115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e82fd7-d270-4830-9d34-faff021dd44e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:23.022348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-4a015b1c-b654-4f50-a2ac-59531bd096af None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:23.024152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4a015b1c-b654-4f50-a2ac-59531bd096af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:23.181354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e6d1f-e05a-45e8-ac54-0b76c3034564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:23.182232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e6d1f-e05a-45e8-ac54-0b76c3034564 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:23.182232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e6d1f-e05a-45e8-ac54-0b76c3034564 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:23.182570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e6d1f-e05a-45e8-ac54-0b76c3034564 admin admin] PATH_INFO: `/v3/projects/ef8f9220e49d4510a60fea95048a59bb/users/7badc1c389f740c0afd7e2080600d20a/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:23.215297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 939/1879] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:23.233832 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e6d1f-e05a-45e8-ac54-0b76c3034564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef8f9220e49d4510a60fea95048a59bb, user_id=7badc1c389f740c0afd7e2080600d20a, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:23.236790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e6d1f-e05a-45e8-ac54-0b76c3034564 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:23.268490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 941/1880] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:22 2022] PUT /identity/v3/projects/ef8f9220e49d4510a60fea95048a59bb/users/7badc1c389f740c0afd7e2080600d20a/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:23.275405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a015b1c-b654-4f50-a2ac-59531bd096af None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:23.309257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a015b1c-b654-4f50-a2ac-59531bd096af None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:23.456499 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e6d1f-e05a-45e8-ac54-0b76c3034564 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:23.460136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9a7e6d1f-e05a-45e8-ac54-0b76c3034564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:23.464639 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94158072-e5ea-4e7e-b74e-c6ad1c718c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:23.467702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94158072-e5ea-4e7e-b74e-c6ad1c718c7c admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:23.468399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94158072-e5ea-4e7e-b74e-c6ad1c718c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:23.469223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94158072-e5ea-4e7e-b74e-c6ad1c718c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:23.476854 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94158072-e5ea-4e7e-b74e-c6ad1c718c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:23.479182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94158072-e5ea-4e7e-b74e-c6ad1c718c7c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:23.498416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 940/1881] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:23.503230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94158072-e5ea-4e7e-b74e-c6ad1c718c7c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:23.531065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94158072-e5ea-4e7e-b74e-c6ad1c718c7c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:23.541767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99892f9d-9c55-4816-800c-c0e2d84bff23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:23.542489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99892f9d-9c55-4816-800c-c0e2d84bff23 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:23.542769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99892f9d-9c55-4816-800c-c0e2d84bff23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:23.543049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99892f9d-9c55-4816-800c-c0e2d84bff23 admin admin] PATH_INFO: `/v3/projects/ef8f9220e49d4510a60fea95048a59bb/users/7badc1c389f740c0afd7e2080600d20a/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:23.547806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99892f9d-9c55-4816-800c-c0e2d84bff23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef8f9220e49d4510a60fea95048a59bb, user_id=7badc1c389f740c0afd7e2080600d20a, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:23.551260 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99892f9d-9c55-4816-800c-c0e2d84bff23 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:23.557594 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:23.571995 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 941/1882] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:23 2022] PUT /identity/v3/projects/ef8f9220e49d4510a60fea95048a59bb/users/7badc1c389f740c0afd7e2080600d20a/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:23.604980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a445318-6f7e-4cb9-b1e3-935914d6d486 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:23.605628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a445318-6f7e-4cb9-b1e3-935914d6d486 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:23.606673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a445318-6f7e-4cb9-b1e3-935914d6d486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:23.647046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e6d1f-e05a-45e8-ac54-0b76c3034564 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:23.689695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0a445318-6f7e-4cb9-b1e3-935914d6d486 None None] MFA Rules not processed for user `7badc1c389f740c0afd7e2080600d20a`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:23.692369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0a445318-6f7e-4cb9-b1e3-935914d6d486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:23.798759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee7317-1e9c-4daf-aa92-9da7b3645b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:23.800570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee7317-1e9c-4daf-aa92-9da7b3645b61 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:23.802205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee7317-1e9c-4daf-aa92-9da7b3645b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:23.803392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee7317-1e9c-4daf-aa92-9da7b3645b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:23.875619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:23.884170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 942/1883] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:23.905223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a445318-6f7e-4cb9-b1e3-935914d6d486 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:23.907651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0a445318-6f7e-4cb9-b1e3-935914d6d486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:23.970062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee7317-1e9c-4daf-aa92-9da7b3645b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:23.981700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee7317-1e9c-4daf-aa92-9da7b3645b61 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:23.999809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 942/1884] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:24.033155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a445318-6f7e-4cb9-b1e3-935914d6d486 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:24.041996 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e3eca-2fdc-4d92-bfaf-4081a90b38a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:24.042794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e3eca-2fdc-4d92-bfaf-4081a90b38a6 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.043330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e3eca-2fdc-4d92-bfaf-4081a90b38a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.043860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e3eca-2fdc-4d92-bfaf-4081a90b38a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.062976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e3eca-2fdc-4d92-bfaf-4081a90b38a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:24.066059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e3eca-2fdc-4d92-bfaf-4081a90b38a6 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:24.096857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 943/1885] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:24.164073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee7317-1e9c-4daf-aa92-9da7b3645b61 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:24.165751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e7ee7317-1e9c-4daf-aa92-9da7b3645b61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:24.181203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee7317-1e9c-4daf-aa92-9da7b3645b61 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:24.187706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565bf320-29f9-49a2-80d1-08a395668c6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:24.188797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bf320-29f9-49a2-80d1-08a395668c6d service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.188797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bf320-29f9-49a2-80d1-08a395668c6d service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.189063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bf320-29f9-49a2-80d1-08a395668c6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.190913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-565bf320-29f9-49a2-80d1-08a395668c6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:24.300590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565bf320-29f9-49a2-80d1-08a395668c6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:24.304097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565bf320-29f9-49a2-80d1-08a395668c6d service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:24.322867 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 943/1886] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:24.459990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e3eca-2fdc-4d92-bfaf-4081a90b38a6 service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:24.462555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e74e3eca-2fdc-4d92-bfaf-4081a90b38a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:24.491168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e3eca-2fdc-4d92-bfaf-4081a90b38a6 service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:24.505708 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fdd8d3-97b7-4829-99c1-6186529ab565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:24.506872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdd8d3-97b7-4829-99c1-6186529ab565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.507440 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdd8d3-97b7-4829-99c1-6186529ab565 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.507933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdd8d3-97b7-4829-99c1-6186529ab565 admin admin] PATH_INFO: `/v3/projects/5a94d540a2ce4857932ec8a6617b05ba` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.511343 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fdd8d3-97b7-4829-99c1-6186529ab565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a94d540a2ce4857932ec8a6617b05ba)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:24.515347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fdd8d3-97b7-4829-99c1-6186529ab565 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:24.637368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-54fdd8d3-97b7-4829-99c1-6186529ab565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a94d540a2ce4857932ec8a6617b05ba', 'request_id': 'req-54fdd8d3-97b7-4829-99c1-6186529ab565'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.661944 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-54fdd8d3-97b7-4829-99c1-6186529ab565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a94d540a2ce4857932ec8a6617b05ba', 'request_id': 'req-54fdd8d3-97b7-4829-99c1-6186529ab565'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.675288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-54fdd8d3-97b7-4829-99c1-6186529ab565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a94d540a2ce4857932ec8a6617b05ba', 'request_id': 'req-54fdd8d3-97b7-4829-99c1-6186529ab565'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.698129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-54fdd8d3-97b7-4829-99c1-6186529ab565 admin admin] The token cache is being invalidate because project 5a94d540a2ce4857932ec8a6617b05ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:24.698573 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-54fdd8d3-97b7-4829-99c1-6186529ab565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.699399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-54fdd8d3-97b7-4829-99c1-6186529ab565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.701242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 944/1887] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:24 2022] DELETE /identity/v3/projects/5a94d540a2ce4857932ec8a6617b05ba => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:24.708407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a324a-9e0d-4b1b-8d67-9b7ad2fc28b3 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.709173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a324a-9e0d-4b1b-8d67-9b7ad2fc28b3 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.709797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a324a-9e0d-4b1b-8d67-9b7ad2fc28b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.757978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-1e5a324a-9e0d-4b1b-8d67-9b7ad2fc28b3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:24.759782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1e5a324a-9e0d-4b1b-8d67-9b7ad2fc28b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:24.808770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:24.867288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 944/1888] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:24.870499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4e3cd-ccc3-4b53-bc38-3087dca21321 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.871066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4e3cd-ccc3-4b53-bc38-3087dca21321 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.871565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4e3cd-ccc3-4b53-bc38-3087dca21321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.918016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-7db4e3cd-ccc3-4b53-bc38-3087dca21321 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:24.920596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7db4e3cd-ccc3-4b53-bc38-3087dca21321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:24.933794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 945/1889] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:24.947887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a324a-9e0d-4b1b-8d67-9b7ad2fc28b3 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:24.956095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a324a-9e0d-4b1b-8d67-9b7ad2fc28b3 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:24.977094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f19b53-64a6-43a5-94a2-e268ae7cec79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:24.982898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f19b53-64a6-43a5-94a2-e268ae7cec79 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.983354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f19b53-64a6-43a5-94a2-e268ae7cec79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.983815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f19b53-64a6-43a5-94a2-e268ae7cec79 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.984705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f19b53-64a6-43a5-94a2-e268ae7cec79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:24.986203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f19b53-64a6-43a5-94a2-e268ae7cec79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:24.988272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f19b53-64a6-43a5-94a2-e268ae7cec79 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.052814 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4e3cd-ccc3-4b53-bc38-3087dca21321 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.059577 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 945/1890] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:25.068006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4e3cd-ccc3-4b53-bc38-3087dca21321 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.086446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75587ad8-3a80-4e56-921a-24e8ad4a71dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.087296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75587ad8-3a80-4e56-921a-24e8ad4a71dd admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.088377 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75587ad8-3a80-4e56-921a-24e8ad4a71dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.089056 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75587ad8-3a80-4e56-921a-24e8ad4a71dd admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.092219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75587ad8-3a80-4e56-921a-24e8ad4a71dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.098144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75587ad8-3a80-4e56-921a-24e8ad4a71dd admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.187707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 946/1891] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:25 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:25.192560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f19b53-64a6-43a5-94a2-e268ae7cec79 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.217587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f19b53-64a6-43a5-94a2-e268ae7cec79 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.322299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d820e5-fcaa-4adc-afb2-b500a911af9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.323390 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d820e5-fcaa-4adc-afb2-b500a911af9c admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.323839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d820e5-fcaa-4adc-afb2-b500a911af9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.324290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d820e5-fcaa-4adc-afb2-b500a911af9c admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.325167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d820e5-fcaa-4adc-afb2-b500a911af9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.328304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d820e5-fcaa-4adc-afb2-b500a911af9c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.332139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c021b3-8216-40ba-a1fb-0f46680e8f40 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.332782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c021b3-8216-40ba-a1fb-0f46680e8f40 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.333332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c021b3-8216-40ba-a1fb-0f46680e8f40 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.335501 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 947/1892] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 09:49:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:49:25.354115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449126b3-9359-43cb-8637-8e5b404e536d None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.354330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449126b3-9359-43cb-8637-8e5b404e536d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.354857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449126b3-9359-43cb-8637-8e5b404e536d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.367259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 946/1893] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:25 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:25.369014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d820e5-fcaa-4adc-afb2-b500a911af9c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.383346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d820e5-fcaa-4adc-afb2-b500a911af9c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.391345 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0ec19-6da6-4c1f-9de8-966ee66f7e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.392310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0ec19-6da6-4c1f-9de8-966ee66f7e6e admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.392859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0ec19-6da6-4c1f-9de8-966ee66f7e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.393248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0ec19-6da6-4c1f-9de8-966ee66f7e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.394194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f0ec19-6da6-4c1f-9de8-966ee66f7e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.397700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f0ec19-6da6-4c1f-9de8-966ee66f7e6e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.402193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-449126b3-9359-43cb-8637-8e5b404e536d None None] MFA Rules not processed for user `ef72bb95992c4a21ae2dbb2a08c51421`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:25.403821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-449126b3-9359-43cb-8637-8e5b404e536d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:25.419313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 947/1894] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:25.420786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0ec19-6da6-4c1f-9de8-966ee66f7e6e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.435477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0ec19-6da6-4c1f-9de8-966ee66f7e6e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.444505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e7265-33ce-4a57-8d05-9affde3eeb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.445541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e7265-33ce-4a57-8d05-9affde3eeb08 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.446026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e7265-33ce-4a57-8d05-9affde3eeb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.446767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e7265-33ce-4a57-8d05-9affde3eeb08 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.448154 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e7265-33ce-4a57-8d05-9affde3eeb08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.452020 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e7265-33ce-4a57-8d05-9affde3eeb08 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.471311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:25.484195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 948/1895] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:25.490032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e7265-33ce-4a57-8d05-9affde3eeb08 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.499786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 948/1896] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 09:49:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:25.508615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e7265-33ce-4a57-8d05-9affde3eeb08 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.516323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394c1a2-2a78-45d9-bb67-3eea620b6167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.517290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394c1a2-2a78-45d9-bb67-3eea620b6167 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.517802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394c1a2-2a78-45d9-bb67-3eea620b6167 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.517905 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449126b3-9359-43cb-8637-8e5b404e536d None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.518255 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394c1a2-2a78-45d9-bb67-3eea620b6167 admin admin] PATH_INFO: `/v3/projects/87fdd1483c9140e692115d0ad7b14780/users/cbfd91621f3340c2a71ce61ba9e0532a/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.522277 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-449126b3-9359-43cb-8637-8e5b404e536d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:25.539305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394c1a2-2a78-45d9-bb67-3eea620b6167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87fdd1483c9140e692115d0ad7b14780, user_id=cbfd91621f3340c2a71ce61ba9e0532a, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.542121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394c1a2-2a78-45d9-bb67-3eea620b6167 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.545810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449126b3-9359-43cb-8637-8e5b404e536d None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.558478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e7510-c4eb-415c-b9bc-5bdc67c7f4e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.559222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e7510-c4eb-415c-b9bc-5bdc67c7f4e9 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.559910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e7510-c4eb-415c-b9bc-5bdc67c7f4e9 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.561141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e7510-c4eb-415c-b9bc-5bdc67c7f4e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.564519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 949/1897] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:25 2022] PUT /identity/v3/projects/87fdd1483c9140e692115d0ad7b14780/users/cbfd91621f3340c2a71ce61ba9e0532a/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:25.587556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e7510-c4eb-415c-b9bc-5bdc67c7f4e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.591035 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e7510-c4eb-415c-b9bc-5bdc67c7f4e9 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.592629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ad38d-c7c7-4b13-a42c-cf6167888e30 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.592922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ad38d-c7c7-4b13-a42c-cf6167888e30 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.593193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ad38d-c7c7-4b13-a42c-cf6167888e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.614061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 949/1898] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:25.687388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-880ad38d-c7c7-4b13-a42c-cf6167888e30 None None] MFA Rules not processed for user `cbfd91621f3340c2a71ce61ba9e0532a`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:25.689281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-880ad38d-c7c7-4b13-a42c-cf6167888e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:25.792557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:25.820683 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e7510-c4eb-415c-b9bc-5bdc67c7f4e9 service nova] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.822283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 950/1899] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:25.823935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3c4e7510-c4eb-415c-b9bc-5bdc67c7f4e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:25.828544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ad38d-c7c7-4b13-a42c-cf6167888e30 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.872859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e7510-c4eb-415c-b9bc-5bdc67c7f4e9 service nova] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.953555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ad38d-c7c7-4b13-a42c-cf6167888e30 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.965046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3093df7-bff3-49c4-95ba-f8e3a8037fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.966485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3093df7-bff3-49c4-95ba-f8e3a8037fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.969493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3093df7-bff3-49c4-95ba-f8e3a8037fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.970310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3093df7-bff3-49c4-95ba-f8e3a8037fa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.971053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3093df7-bff3-49c4-95ba-f8e3a8037fa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.974010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3093df7-bff3-49c4-95ba-f8e3a8037fa0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:26.007266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5fe22-2911-4def-b109-be4d263068d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:26.007955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5fe22-2911-4def-b109-be4d263068d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.008293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5fe22-2911-4def-b109-be4d263068d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.008577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5fe22-2911-4def-b109-be4d263068d5 admin admin] PATH_INFO: `/v3/users/d85ee7e6a90f4e8a8bfd7da370140ad9` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.009954 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5fe22-2911-4def-b109-be4d263068d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d85ee7e6a90f4e8a8bfd7da370140ad9)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:26.012170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5fe22-2911-4def-b109-be4d263068d5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:26.031300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 951/1900] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:25 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:26.036731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3093df7-bff3-49c4-95ba-f8e3a8037fa0 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:26.124991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-aef5fe22-2911-4def-b109-be4d263068d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd85ee7e6a90f4e8a8bfd7da370140ad9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:26.146815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-aef5fe22-2911-4def-b109-be4d263068d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd85ee7e6a90f4e8a8bfd7da370140ad9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:26.148008 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-aef5fe22-2911-4def-b109-be4d263068d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd85ee7e6a90f4e8a8bfd7da370140ad9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:26.148738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-aef5fe22-2911-4def-b109-be4d263068d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd85ee7e6a90f4e8a8bfd7da370140ad9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:26.159651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-aef5fe22-2911-4def-b109-be4d263068d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd85ee7e6a90f4e8a8bfd7da370140ad9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:26.202889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 950/1901] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:25 2022] DELETE /identity/v3/users/d85ee7e6a90f4e8a8bfd7da370140ad9 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:26.210175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be702a02-a744-4a47-938c-18cd937d09c3 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.211567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be702a02-a744-4a47-938c-18cd937d09c3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.212728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be702a02-a744-4a47-938c-18cd937d09c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.262273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3093df7-bff3-49c4-95ba-f8e3a8037fa0 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:26.294162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-be702a02-a744-4a47-938c-18cd937d09c3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:26.296224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-be702a02-a744-4a47-938c-18cd937d09c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:26.448365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:26.483206 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 951/1902] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:26.524943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dd3a3-49ae-44c2-876c-82606900b5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:26.525774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dd3a3-49ae-44c2-876c-82606900b5a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.526166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dd3a3-49ae-44c2-876c-82606900b5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.527679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dd3a3-49ae-44c2-876c-82606900b5a3 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.529446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be702a02-a744-4a47-938c-18cd937d09c3 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:26.531416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-be702a02-a744-4a47-938c-18cd937d09c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:26.545414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dd3a3-49ae-44c2-876c-82606900b5a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:26.548108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dd3a3-49ae-44c2-876c-82606900b5a3 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:26.610735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 952/1903] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:26 2022] POST /identity/v3/users => generated 288 bytes in 576 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:26.611915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dd3a3-49ae-44c2-876c-82606900b5a3 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:26.615697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5c2dd3a3-49ae-44c2-876c-82606900b5a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:26.655746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be702a02-a744-4a47-938c-18cd937d09c3 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:26.661961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4658bb-1274-421a-895f-0dd6522c8063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:26.663432 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4658bb-1274-421a-895f-0dd6522c8063 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.665097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4658bb-1274-421a-895f-0dd6522c8063 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.665651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4658bb-1274-421a-895f-0dd6522c8063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.670956 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:26.686298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4658bb-1274-421a-895f-0dd6522c8063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:26.688476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4658bb-1274-421a-895f-0dd6522c8063 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:26.703915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 952/1904] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:26.704999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4658bb-1274-421a-895f-0dd6522c8063 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:26.707776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ac4658bb-1274-421a-895f-0dd6522c8063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:26.728398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4658bb-1274-421a-895f-0dd6522c8063 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:26.735809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21178a95-a135-4f79-89be-bcb8f922620f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:26.737051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21178a95-a135-4f79-89be-bcb8f922620f admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.737655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21178a95-a135-4f79-89be-bcb8f922620f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.738264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21178a95-a135-4f79-89be-bcb8f922620f admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.739718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21178a95-a135-4f79-89be-bcb8f922620f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:26.742949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21178a95-a135-4f79-89be-bcb8f922620f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:26.761667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 953/1905] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:26.763038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21178a95-a135-4f79-89be-bcb8f922620f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:26.783271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21178a95-a135-4f79-89be-bcb8f922620f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:26.790187 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133b06a-9d9f-4a69-a7a6-e82344cbe24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:26.791061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133b06a-9d9f-4a69-a7a6-e82344cbe24e admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.791522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133b06a-9d9f-4a69-a7a6-e82344cbe24e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.791979 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133b06a-9d9f-4a69-a7a6-e82344cbe24e admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.792821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133b06a-9d9f-4a69-a7a6-e82344cbe24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:26.794648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133b06a-9d9f-4a69-a7a6-e82344cbe24e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:26.810880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 954/1906] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:26.816224 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dd3a3-49ae-44c2-876c-82606900b5a3 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:26.823114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133b06a-9d9f-4a69-a7a6-e82344cbe24e admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:26.831562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e08c5-acfb-4219-9d69-6625cd9b422f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:26.832838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e08c5-acfb-4219-9d69-6625cd9b422f service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.833292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e08c5-acfb-4219-9d69-6625cd9b422f service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.833576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e08c5-acfb-4219-9d69-6625cd9b422f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.841233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7f4e08c5-acfb-4219-9d69-6625cd9b422f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:26.845780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133b06a-9d9f-4a69-a7a6-e82344cbe24e admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:26.858500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509f33d-0498-49d0-b542-e4b304c9f025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:26.859513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509f33d-0498-49d0-b542-e4b304c9f025 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.859938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509f33d-0498-49d0-b542-e4b304c9f025 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.860319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509f33d-0498-49d0-b542-e4b304c9f025 admin admin] PATH_INFO: `/v3/projects/ab884a4da54646aa9079d820192f21b1/users/e9e0eefa163844aa8a98b19ab8d5c6c4/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.900316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509f33d-0498-49d0-b542-e4b304c9f025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab884a4da54646aa9079d820192f21b1, user_id=e9e0eefa163844aa8a98b19ab8d5c6c4, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:26.905901 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509f33d-0498-49d0-b542-e4b304c9f025 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:26.936349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 955/1907] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:26 2022] PUT /identity/v3/projects/ab884a4da54646aa9079d820192f21b1/users/e9e0eefa163844aa8a98b19ab8d5c6c4/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:26.959160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f1737-6d95-486f-9252-a7c76ae1ed3d None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.959838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f1737-6d95-486f-9252-a7c76ae1ed3d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.960296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f1737-6d95-486f-9252-a7c76ae1ed3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:27.037416 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e08c5-acfb-4219-9d69-6625cd9b422f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:27.042978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e08c5-acfb-4219-9d69-6625cd9b422f service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:27.062276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 953/1908] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:27.087575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-294f1737-6d95-486f-9252-a7c76ae1ed3d None None] MFA Rules not processed for user `e9e0eefa163844aa8a98b19ab8d5c6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:27.089952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-294f1737-6d95-486f-9252-a7c76ae1ed3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:27.205644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 956/1909] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:27.207000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e08c5-acfb-4219-9d69-6625cd9b422f service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:27.228687 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e08c5-acfb-4219-9d69-6625cd9b422f service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:27.349254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14925d5e-8f13-40f4-86d7-bcadfb47a0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:27.351590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14925d5e-8f13-40f4-86d7-bcadfb47a0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:27.352074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14925d5e-8f13-40f4-86d7-bcadfb47a0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:27.352682 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14925d5e-8f13-40f4-86d7-bcadfb47a0f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:27.354911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14925d5e-8f13-40f4-86d7-bcadfb47a0f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:27.360934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14925d5e-8f13-40f4-86d7-bcadfb47a0f8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:27.450528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 954/1910] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:27 2022] POST /identity/v3/projects => generated 377 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:27.456816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f1737-6d95-486f-9252-a7c76ae1ed3d None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:27.490471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f1737-6d95-486f-9252-a7c76ae1ed3d None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:27.594333 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:27.646517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a8507-8d18-47e5-8448-43ba54a4f53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:27.647361 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a8507-8d18-47e5-8448-43ba54a4f53e admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:27.647743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a8507-8d18-47e5-8448-43ba54a4f53e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:27.648118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a8507-8d18-47e5-8448-43ba54a4f53e admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:27.648880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a8507-8d18-47e5-8448-43ba54a4f53e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:27.651264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a8507-8d18-47e5-8448-43ba54a4f53e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:27.684938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 957/1911] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:27 2022] POST /identity/v3/users => generated 295 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:27.691359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14925d5e-8f13-40f4-86d7-bcadfb47a0f8 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:27.716448 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14925d5e-8f13-40f4-86d7-bcadfb47a0f8 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:27.734441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745cc3-dda2-4172-b526-e51de9e72c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:27.735526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745cc3-dda2-4172-b526-e51de9e72c94 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:27.735708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745cc3-dda2-4172-b526-e51de9e72c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:27.736815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745cc3-dda2-4172-b526-e51de9e72c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:27.736815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0745cc3-dda2-4172-b526-e51de9e72c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:27.739642 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0745cc3-dda2-4172-b526-e51de9e72c94 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:27.762364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 955/1912] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:27.762937 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a8507-8d18-47e5-8448-43ba54a4f53e admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:27.778311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a8507-8d18-47e5-8448-43ba54a4f53e admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:27.790246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890996ac-947e-4fe9-91d9-be2670dde06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:27.790563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890996ac-947e-4fe9-91d9-be2670dde06e admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:27.790847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890996ac-947e-4fe9-91d9-be2670dde06e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:27.791222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890996ac-947e-4fe9-91d9-be2670dde06e admin admin] PATH_INFO: `/v3/projects/6fe35eec17a94e8ba8670d5f08f36ec4/users/cc05ba3ebaaa471b80a4a0d14c91fcf4/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:27.828188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890996ac-947e-4fe9-91d9-be2670dde06e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe35eec17a94e8ba8670d5f08f36ec4, user_id=cc05ba3ebaaa471b80a4a0d14c91fcf4, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:27.831692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890996ac-947e-4fe9-91d9-be2670dde06e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:27.852694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 958/1913] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:27 2022] PUT /identity/v3/projects/6fe35eec17a94e8ba8670d5f08f36ec4/users/cc05ba3ebaaa471b80a4a0d14c91fcf4/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:27.858347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745cc3-dda2-4172-b526-e51de9e72c94 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:27.880252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890996ac-947e-4fe9-91d9-be2670dde06e admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:27.884822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745cc3-dda2-4172-b526-e51de9e72c94 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:27.889123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-890996ac-947e-4fe9-91d9-be2670dde06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:28.027882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890996ac-947e-4fe9-91d9-be2670dde06e admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:28.030521 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:28.036685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:28.037681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:28.038074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:28.038446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] PATH_INFO: `/v3/projects/04f9057993004696adf85b46b8cc2ff2` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:28.041041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f9057993004696adf85b46b8cc2ff2)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:28.044840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:28.085390 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0684c0-8d35-457c-8565-bbe1f0ffbc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:28.086314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0684c0-8d35-457c-8565-bbe1f0ffbc1b admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:28.087495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0684c0-8d35-457c-8565-bbe1f0ffbc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:28.087495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0684c0-8d35-457c-8565-bbe1f0ffbc1b admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:28.089179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0684c0-8d35-457c-8565-bbe1f0ffbc1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:28.091549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0684c0-8d35-457c-8565-bbe1f0ffbc1b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:28.124699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 956/1914] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:28.130228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0684c0-8d35-457c-8565-bbe1f0ffbc1b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:28.131100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f9057993004696adf85b46b8cc2ff2', 'request_id': 'req-9d32cd09-b283-472f-b96a-e05c73ee4992'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:28.148758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f9057993004696adf85b46b8cc2ff2', 'request_id': 'req-9d32cd09-b283-472f-b96a-e05c73ee4992'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:28.155052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0684c0-8d35-457c-8565-bbe1f0ffbc1b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:28.158739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f9057993004696adf85b46b8cc2ff2', 'request_id': 'req-9d32cd09-b283-472f-b96a-e05c73ee4992'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:28.178861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] The token cache is being invalidate because project 04f9057993004696adf85b46b8cc2ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:28.179358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:28.180223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:28.182122 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 959/1915] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:27 2022] DELETE /identity/v3/projects/04f9057993004696adf85b46b8cc2ff2 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:28.186534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:28.188159 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:28.284420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:28.326284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18bfb2-0aab-4c0f-a692-cfd83d8ec134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:28.326284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18bfb2-0aab-4c0f-a692-cfd83d8ec134 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:28.326284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18bfb2-0aab-4c0f-a692-cfd83d8ec134 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:28.326284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18bfb2-0aab-4c0f-a692-cfd83d8ec134 admin admin] PATH_INFO: `/v3/projects/6fe35eec17a94e8ba8670d5f08f36ec4/users/cc05ba3ebaaa471b80a4a0d14c91fcf4/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:28.332264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18bfb2-0aab-4c0f-a692-cfd83d8ec134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe35eec17a94e8ba8670d5f08f36ec4, user_id=cc05ba3ebaaa471b80a4a0d14c91fcf4, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:28.340673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18bfb2-0aab-4c0f-a692-cfd83d8ec134 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:28.387167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 957/1916] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:28 2022] PUT /identity/v3/projects/6fe35eec17a94e8ba8670d5f08f36ec4/users/cc05ba3ebaaa471b80a4a0d14c91fcf4/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:28.414735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32cd09-b283-472f-b96a-e05c73ee4992 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:28.414947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdb5b7-40ad-47ce-8395-d046cc808c85 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:28.415731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdb5b7-40ad-47ce-8395-d046cc808c85 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:28.416363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdb5b7-40ad-47ce-8395-d046cc808c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:28.520588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-94bdb5b7-40ad-47ce-8395-d046cc808c85 None None] MFA Rules not processed for user `cc05ba3ebaaa471b80a4a0d14c91fcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:28.522764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-94bdb5b7-40ad-47ce-8395-d046cc808c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:28.561289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:28.561953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:28.562231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:28.563289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] PATH_INFO: `/v3/users/fcf76cf19b1547dca933307b675213d9` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:28.565018 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcf76cf19b1547dca933307b675213d9)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:28.568382 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:28.765811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcf76cf19b1547dca933307b675213d9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:28.790381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcf76cf19b1547dca933307b675213d9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:28.791445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcf76cf19b1547dca933307b675213d9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:28.792188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcf76cf19b1547dca933307b675213d9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:28.812644 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 958/1917] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:28.814349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdb5b7-40ad-47ce-8395-d046cc808c85 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:28.818689 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-94bdb5b7-40ad-47ce-8395-d046cc808c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:28.840976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcf76cf19b1547dca933307b675213d9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:28.864764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 960/1918] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:28 2022] DELETE /identity/v3/users/fcf76cf19b1547dca933307b675213d9 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:28.865703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:28.872195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:28.886017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdb5b7-40ad-47ce-8395-d046cc808c85 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:29.029013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:29.030278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:29.030948 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:29.031500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] PATH_INFO: `/v3/users/364a0a76f6ac45deb80e1ef8257f962f` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:29.033402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=364a0a76f6ac45deb80e1ef8257f962f)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:29.038514 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:29.041629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:29.054415 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357684bb-0a0e-4a2b-ac31-f71696dc8d42 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:29.059634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5486d-d54b-41fe-9670-f9f6006ba988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:29.060460 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5486d-d54b-41fe-9670-f9f6006ba988 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:29.060805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5486d-d54b-41fe-9670-f9f6006ba988 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:29.061145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5486d-d54b-41fe-9670-f9f6006ba988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:29.063375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-71a5486d-d54b-41fe-9670-f9f6006ba988 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:29.120281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:29.201917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364a0a76f6ac45deb80e1ef8257f962f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:29.220007 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364a0a76f6ac45deb80e1ef8257f962f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:29.220775 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364a0a76f6ac45deb80e1ef8257f962f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:29.232541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5486d-d54b-41fe-9670-f9f6006ba988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:29.234371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5486d-d54b-41fe-9670-f9f6006ba988 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:29.250563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 961/1919] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:29.265490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364a0a76f6ac45deb80e1ef8257f962f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:29.267347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364a0a76f6ac45deb80e1ef8257f962f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:29.309632 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 959/1920] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:28 2022] DELETE /identity/v3/users/364a0a76f6ac45deb80e1ef8257f962f => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:29.318970 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7e9e6-b60a-4889-81fb-0c47d5691232 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:29.319484 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7e9e6-b60a-4889-81fb-0c47d5691232 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:29.319852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7e9e6-b60a-4889-81fb-0c47d5691232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:29.367187 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-07c7e9e6-b60a-4889-81fb-0c47d5691232 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:29.372410 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-07c7e9e6-b60a-4889-81fb-0c47d5691232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:29.552810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 962/1921] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:29.572921 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:29.575029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:29.666228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7e9e6-b60a-4889-81fb-0c47d5691232 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:29.668249 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-07c7e9e6-b60a-4889-81fb-0c47d5691232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:29.700656 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7e9e6-b60a-4889-81fb-0c47d5691232 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:29.711936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7566b-d56f-4317-a531-28fb0370261b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:29.713473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7566b-d56f-4317-a531-28fb0370261b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:29.714379 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7566b-d56f-4317-a531-28fb0370261b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:29.716851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7566b-d56f-4317-a531-28fb0370261b admin admin] PATH_INFO: `/v3/projects/c5ba332e27424d1bb34a7b79734090f3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:29.719948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7566b-d56f-4317-a531-28fb0370261b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5ba332e27424d1bb34a7b79734090f3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:29.724122 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7566b-d56f-4317-a531-28fb0370261b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:29.792226 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca35e8-3cc0-46b1-9eb7-33f00f6b9c37 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:29.803741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8f8b9-7f09-4b41-9cb3-23505e4abf31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:29.805534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f8b9-7f09-4b41-9cb3-23505e4abf31 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:29.806432 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f8b9-7f09-4b41-9cb3-23505e4abf31 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:29.821242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f8b9-7f09-4b41-9cb3-23505e4abf31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:29.850940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-62b7566b-d56f-4317-a531-28fb0370261b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5ba332e27424d1bb34a7b79734090f3', 'request_id': 'req-62b7566b-d56f-4317-a531-28fb0370261b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:29.873558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-62b7566b-d56f-4317-a531-28fb0370261b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5ba332e27424d1bb34a7b79734090f3', 'request_id': 'req-62b7566b-d56f-4317-a531-28fb0370261b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:29.883996 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-62b7566b-d56f-4317-a531-28fb0370261b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5ba332e27424d1bb34a7b79734090f3', 'request_id': 'req-62b7566b-d56f-4317-a531-28fb0370261b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:29.904267 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8f8b9-7f09-4b41-9cb3-23505e4abf31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:29.908992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8f8b9-7f09-4b41-9cb3-23505e4abf31 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:29.912960 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-62b7566b-d56f-4317-a531-28fb0370261b admin admin] The token cache is being invalidate because project c5ba332e27424d1bb34a7b79734090f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:29.913312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-62b7566b-d56f-4317-a531-28fb0370261b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:29.924548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-62b7566b-d56f-4317-a531-28fb0370261b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:29.924548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 963/1922] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:29 2022] DELETE /identity/v3/projects/c5ba332e27424d1bb34a7b79734090f3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:29.930097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d835b3-d73c-430f-8fca-cd5cd572ebd1 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:29.930538 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d835b3-d73c-430f-8fca-cd5cd572ebd1 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:29.930962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d835b3-d73c-430f-8fca-cd5cd572ebd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:29.957941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 960/1923] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:29.999789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-80d835b3-d73c-430f-8fca-cd5cd572ebd1 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:30.001335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-80d835b3-d73c-430f-8fca-cd5cd572ebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:30.126055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:30.227515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 964/1924] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:30.238970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf20e83-7274-49bb-be8c-23687d52cddf None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:30.238970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf20e83-7274-49bb-be8c-23687d52cddf None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:30.238970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf20e83-7274-49bb-be8c-23687d52cddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:30.271570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-bcf20e83-7274-49bb-be8c-23687d52cddf None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:30.272938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-bcf20e83-7274-49bb-be8c-23687d52cddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:30.283100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 961/1925] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:30.289752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d835b3-d73c-430f-8fca-cd5cd572ebd1 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:30.309301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d835b3-d73c-430f-8fca-cd5cd572ebd1 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:30.321542 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca33f5-fdd6-474b-8522-9db8d6bd32a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:30.322171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca33f5-fdd6-474b-8522-9db8d6bd32a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:30.322442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca33f5-fdd6-474b-8522-9db8d6bd32a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:30.322732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca33f5-fdd6-474b-8522-9db8d6bd32a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:30.323491 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca33f5-fdd6-474b-8522-9db8d6bd32a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:30.324571 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca33f5-fdd6-474b-8522-9db8d6bd32a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:30.328453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca33f5-fdd6-474b-8522-9db8d6bd32a5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:30.355188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 965/1926] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:49:30.358319 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf20e83-7274-49bb-be8c-23687d52cddf None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:30.400762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf20e83-7274-49bb-be8c-23687d52cddf None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:30.410589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0155f-ec5a-4531-855d-0139be08f9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:30.412152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0155f-ec5a-4531-855d-0139be08f9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:30.412599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0155f-ec5a-4531-855d-0139be08f9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:30.413029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0155f-ec5a-4531-855d-0139be08f9d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:30.413707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0155f-ec5a-4531-855d-0139be08f9d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:30.417600 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0155f-ec5a-4531-855d-0139be08f9d0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:30.464104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 962/1927] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:30 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:30.469380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca33f5-fdd6-474b-8522-9db8d6bd32a5 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:30.494878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca33f5-fdd6-474b-8522-9db8d6bd32a5 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:30.611233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85abdf-d369-4663-adec-253a75bbe81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:30.612024 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85abdf-d369-4663-adec-253a75bbe81e admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:30.612312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85abdf-d369-4663-adec-253a75bbe81e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:30.612587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85abdf-d369-4663-adec-253a75bbe81e admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:30.613906 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85abdf-d369-4663-adec-253a75bbe81e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:30.616876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85abdf-d369-4663-adec-253a75bbe81e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:30.679463 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 966/1928] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:30 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:30.682848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0155f-ec5a-4531-855d-0139be08f9d0 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:30.695794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85abdf-d369-4663-adec-253a75bbe81e admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:30.699161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0155f-ec5a-4531-855d-0139be08f9d0 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:30.704097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3d85abdf-d369-4663-adec-253a75bbe81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:30.714079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da207f-d2d7-4aad-932e-42a7d0a3fa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:30.715264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da207f-d2d7-4aad-932e-42a7d0a3fa89 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:30.717214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da207f-d2d7-4aad-932e-42a7d0a3fa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:30.717635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da207f-d2d7-4aad-932e-42a7d0a3fa89 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:30.719948 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da207f-d2d7-4aad-932e-42a7d0a3fa89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:30.721717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da207f-d2d7-4aad-932e-42a7d0a3fa89 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:30.729670 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85abdf-d369-4663-adec-253a75bbe81e admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:30.742464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:30.743412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:30.743762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:30.744987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] PATH_INFO: `/v3/projects/1ddb7c87c39a4c148da5893b62f94ad3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:30.746370 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ddb7c87c39a4c148da5893b62f94ad3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:30.748637 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:30.781958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 963/1929] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:30.788700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da207f-d2d7-4aad-932e-42a7d0a3fa89 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:30.823269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da207f-d2d7-4aad-932e-42a7d0a3fa89 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:30.838772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ddb7c87c39a4c148da5893b62f94ad3', 'request_id': 'req-e1138b65-631f-48e2-b3a9-cbc895b91d28'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:30.860183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ddb7c87c39a4c148da5893b62f94ad3', 'request_id': 'req-e1138b65-631f-48e2-b3a9-cbc895b91d28'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:30.870887 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ddb7c87c39a4c148da5893b62f94ad3', 'request_id': 'req-e1138b65-631f-48e2-b3a9-cbc895b91d28'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:30.889384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] The token cache is being invalidate because project 1ddb7c87c39a4c148da5893b62f94ad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:30.889888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:30.891681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:30.892178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:30.899647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 967/1930] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:30 2022] DELETE /identity/v3/projects/1ddb7c87c39a4c148da5893b62f94ad3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:30.901882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:30.904626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:30.921725 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe15de-f720-48e4-b0f8-c1eb381de3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:30.922782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe15de-f720-48e4-b0f8-c1eb381de3df admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:30.923303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe15de-f720-48e4-b0f8-c1eb381de3df admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:30.923723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe15de-f720-48e4-b0f8-c1eb381de3df admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:30.924538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe15de-f720-48e4-b0f8-c1eb381de3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:30.927329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe15de-f720-48e4-b0f8-c1eb381de3df admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:30.941007 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 964/1931] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:30.944429 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe15de-f720-48e4-b0f8-c1eb381de3df admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:30.947013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-06fe15de-f720-48e4-b0f8-c1eb381de3df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:30.964482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe15de-f720-48e4-b0f8-c1eb381de3df admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:30.970365 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:30.971474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:30.971944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:30.972898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] PATH_INFO: `/v3/users/ac756f53a5114329b649c14fdfbbad02` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:30.974743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac756f53a5114329b649c14fdfbbad02)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:30.977588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:31.030709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:31.049099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac756f53a5114329b649c14fdfbbad02'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:31.075262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac756f53a5114329b649c14fdfbbad02'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:31.077326 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac756f53a5114329b649c14fdfbbad02'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:31.116441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac756f53a5114329b649c14fdfbbad02'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:31.117773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac756f53a5114329b649c14fdfbbad02'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:31.119208 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1138b65-631f-48e2-b3a9-cbc895b91d28 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:31.131255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffb050-d595-4ee2-98a9-6f1571767a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:31.131990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffb050-d595-4ee2-98a9-6f1571767a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:31.132355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffb050-d595-4ee2-98a9-6f1571767a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:31.132635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffb050-d595-4ee2-98a9-6f1571767a3a admin admin] PATH_INFO: `/v3/users/1656370fe93a42de8791f2df634ea363` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:31.134322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffb050-d595-4ee2-98a9-6f1571767a3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1656370fe93a42de8791f2df634ea363)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:31.146741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 965/1932] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:30 2022] DELETE /identity/v3/users/ac756f53a5114329b649c14fdfbbad02 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:31.146741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:31.146741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:31.150304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffb050-d595-4ee2-98a9-6f1571767a3a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:31.270926 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-10ffb050-d595-4ee2-98a9-6f1571767a3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1656370fe93a42de8791f2df634ea363'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:31.320888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-10ffb050-d595-4ee2-98a9-6f1571767a3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1656370fe93a42de8791f2df634ea363'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:31.329668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-10ffb050-d595-4ee2-98a9-6f1571767a3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1656370fe93a42de8791f2df634ea363'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:31.332332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-10ffb050-d595-4ee2-98a9-6f1571767a3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1656370fe93a42de8791f2df634ea363'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:31.362524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc4f3c-429d-4bf4-b6d6-e47750a3b115 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:31.367364 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-10ffb050-d595-4ee2-98a9-6f1571767a3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1656370fe93a42de8791f2df634ea363'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:31.378608 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630d2b1-9cf2-40ee-a7ec-35582d7976a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:31.380021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630d2b1-9cf2-40ee-a7ec-35582d7976a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:31.380324 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630d2b1-9cf2-40ee-a7ec-35582d7976a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:31.380608 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630d2b1-9cf2-40ee-a7ec-35582d7976a5 admin admin] PATH_INFO: `/v3/projects/2c3e3d29f9cc46afbf7b6800a1ebe05a/users/67cce6bbc10c4b199dcc199b87c4c0cf/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:31.419087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 968/1933] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:30 2022] DELETE /identity/v3/users/1656370fe93a42de8791f2df634ea363 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:31.421829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0b555-adcd-48d1-9e69-24a912aceb46 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:31.422451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0b555-adcd-48d1-9e69-24a912aceb46 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:31.423029 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0b555-adcd-48d1-9e69-24a912aceb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:31.430386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630d2b1-9cf2-40ee-a7ec-35582d7976a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c3e3d29f9cc46afbf7b6800a1ebe05a, user_id=67cce6bbc10c4b199dcc199b87c4c0cf, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:31.437906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630d2b1-9cf2-40ee-a7ec-35582d7976a5 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:31.466952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-15c0b555-adcd-48d1-9e69-24a912aceb46 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:31.469501 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-15c0b555-adcd-48d1-9e69-24a912aceb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:31.472090 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 966/1934] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:31 2022] PUT /identity/v3/projects/2c3e3d29f9cc46afbf7b6800a1ebe05a/users/67cce6bbc10c4b199dcc199b87c4c0cf/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:31.480204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630d2b1-9cf2-40ee-a7ec-35582d7976a5 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:31.499902 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6630d2b1-9cf2-40ee-a7ec-35582d7976a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:31.607412 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:31.647430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630d2b1-9cf2-40ee-a7ec-35582d7976a5 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:31.656332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413945ae-baff-41de-afc8-64d7c680f238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:31.657026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413945ae-baff-41de-afc8-64d7c680f238 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:31.657370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413945ae-baff-41de-afc8-64d7c680f238 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:31.657650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413945ae-baff-41de-afc8-64d7c680f238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:31.661799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-413945ae-baff-41de-afc8-64d7c680f238 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:31.668544 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 969/1935] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:31.670074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d15be9-9638-488a-a468-6fab56ed0542 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:31.670419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d15be9-9638-488a-a468-6fab56ed0542 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:31.670699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d15be9-9638-488a-a468-6fab56ed0542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:31.707923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-19d15be9-9638-488a-a468-6fab56ed0542 None None] MFA Rules not processed for user `67cce6bbc10c4b199dcc199b87c4c0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:31.709435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-19d15be9-9638-488a-a468-6fab56ed0542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:31.780604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:31.812925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 970/1936] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:31.813831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d15be9-9638-488a-a468-6fab56ed0542 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:31.818760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-19d15be9-9638-488a-a468-6fab56ed0542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:31.820238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413945ae-baff-41de-afc8-64d7c680f238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:31.822078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413945ae-baff-41de-afc8-64d7c680f238 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:31.861911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 967/1937] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:31.866069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413945ae-baff-41de-afc8-64d7c680f238 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:31.872682 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-413945ae-baff-41de-afc8-64d7c680f238 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:31.936121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413945ae-baff-41de-afc8-64d7c680f238 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:31.955166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fdd73-1fff-4342-aea8-65ad7c99f1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:31.956665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fdd73-1fff-4342-aea8-65ad7c99f1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:31.958594 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fdd73-1fff-4342-aea8-65ad7c99f1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:31.960375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fdd73-1fff-4342-aea8-65ad7c99f1b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:31.961230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088fdd73-1fff-4342-aea8-65ad7c99f1b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:31.967907 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088fdd73-1fff-4342-aea8-65ad7c99f1b0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:31.992695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d15be9-9638-488a-a468-6fab56ed0542 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:31.997660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89923266-5b36-4582-82b5-38e67b404fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:31.998288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89923266-5b36-4582-82b5-38e67b404fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:31.998799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89923266-5b36-4582-82b5-38e67b404fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:31.999071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89923266-5b36-4582-82b5-38e67b404fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:32.012308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89923266-5b36-4582-82b5-38e67b404fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:32.013790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89923266-5b36-4582-82b5-38e67b404fb9 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:32.026786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 971/1938] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:32.047588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 968/1939] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:31 2022] POST /identity/v3/projects => generated 363 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:32.051835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89923266-5b36-4582-82b5-38e67b404fb9 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:32.062312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89923266-5b36-4582-82b5-38e67b404fb9 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:32.194889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4ac49-0f9d-4798-ad67-136cf9854ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:32.195332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4ac49-0f9d-4798-ad67-136cf9854ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:32.195909 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4ac49-0f9d-4798-ad67-136cf9854ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:32.196362 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4ac49-0f9d-4798-ad67-136cf9854ea4 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:32.197167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4ac49-0f9d-4798-ad67-136cf9854ea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:32.200038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4ac49-0f9d-4798-ad67-136cf9854ea4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:32.241909 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 972/1940] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:32 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:32.247077 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fdd73-1fff-4342-aea8-65ad7c99f1b0 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:32.286122 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fdd73-1fff-4342-aea8-65ad7c99f1b0 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:32.305056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea6caf-c762-4f5a-94a1-d00669418c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:32.306160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea6caf-c762-4f5a-94a1-d00669418c54 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:32.309173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea6caf-c762-4f5a-94a1-d00669418c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:32.309737 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea6caf-c762-4f5a-94a1-d00669418c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:32.313930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea6caf-c762-4f5a-94a1-d00669418c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:32.324262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea6caf-c762-4f5a-94a1-d00669418c54 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:32.383295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 969/1941] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:32.388849 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4ac49-0f9d-4798-ad67-136cf9854ea4 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:32.462504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4ac49-0f9d-4798-ad67-136cf9854ea4 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:32.481425 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402ea24-a4d2-4b47-8568-8f78ce91caf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:32.482128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402ea24-a4d2-4b47-8568-8f78ce91caf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:32.484617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402ea24-a4d2-4b47-8568-8f78ce91caf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:32.485172 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402ea24-a4d2-4b47-8568-8f78ce91caf2 admin admin] PATH_INFO: `/v3/projects/63fea73a2d9a4735843579be6b8b0ec4/users/4734c778339b46879925b6c2adbbbc08/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:32.489310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea6caf-c762-4f5a-94a1-d00669418c54 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:32.492582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-92ea6caf-c762-4f5a-94a1-d00669418c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:32.532919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402ea24-a4d2-4b47-8568-8f78ce91caf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63fea73a2d9a4735843579be6b8b0ec4, user_id=4734c778339b46879925b6c2adbbbc08, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:32.541154 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402ea24-a4d2-4b47-8568-8f78ce91caf2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:32.569435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 973/1942] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:32 2022] PUT /identity/v3/projects/63fea73a2d9a4735843579be6b8b0ec4/users/4734c778339b46879925b6c2adbbbc08/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:32.570865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402ea24-a4d2-4b47-8568-8f78ce91caf2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:32.588543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea6caf-c762-4f5a-94a1-d00669418c54 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:32.689399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:32.695497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402ea24-a4d2-4b47-8568-8f78ce91caf2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:32.715441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a729ee-632a-4360-86ae-3869e5417c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:32.716087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a729ee-632a-4360-86ae-3869e5417c5a admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:32.717555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a729ee-632a-4360-86ae-3869e5417c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:32.717555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a729ee-632a-4360-86ae-3869e5417c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:32.717555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a729ee-632a-4360-86ae-3869e5417c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:32.718964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a729ee-632a-4360-86ae-3869e5417c5a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:32.750770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 974/1943] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:32.751799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a729ee-632a-4360-86ae-3869e5417c5a admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:32.778122 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a729ee-632a-4360-86ae-3869e5417c5a admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:32.784738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afadfdb-990a-4e87-80d6-a84b6916d20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:32.785876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afadfdb-990a-4e87-80d6-a84b6916d20d admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:32.786202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afadfdb-990a-4e87-80d6-a84b6916d20d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:32.787007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afadfdb-990a-4e87-80d6-a84b6916d20d admin admin] PATH_INFO: `/v3/projects/63fea73a2d9a4735843579be6b8b0ec4/users/4734c778339b46879925b6c2adbbbc08/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:32.789211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afadfdb-990a-4e87-80d6-a84b6916d20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63fea73a2d9a4735843579be6b8b0ec4, user_id=4734c778339b46879925b6c2adbbbc08, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:32.793401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afadfdb-990a-4e87-80d6-a84b6916d20d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:32.799162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:32.799993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:32.800473 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:32.801069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] PATH_INFO: `/v3/projects/89c1bd62ecad464da702ab840d152b13` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:32.802984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] RBAC: Authorizing `identity:delete_project(project_id=89c1bd62ecad464da702ab840d152b13)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:32.805539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:32.819331 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 975/1944] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:32 2022] PUT /identity/v3/projects/63fea73a2d9a4735843579be6b8b0ec4/users/4734c778339b46879925b6c2adbbbc08/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:32.823776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90cb02-f4a7-4402-9397-8965fd5ace92 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:32.823776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90cb02-f4a7-4402-9397-8965fd5ace92 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:32.823776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90cb02-f4a7-4402-9397-8965fd5ace92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:32.865271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-6a90cb02-f4a7-4402-9397-8965fd5ace92 None None] MFA Rules not processed for user `4734c778339b46879925b6c2adbbbc08`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:32.867030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6a90cb02-f4a7-4402-9397-8965fd5ace92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:32.940554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89c1bd62ecad464da702ab840d152b13', 'request_id': 'req-d9143e86-20ca-42f0-8848-4b5e1d98715c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:32.952225 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89c1bd62ecad464da702ab840d152b13', 'request_id': 'req-d9143e86-20ca-42f0-8848-4b5e1d98715c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:32.956040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:32.970445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89c1bd62ecad464da702ab840d152b13', 'request_id': 'req-d9143e86-20ca-42f0-8848-4b5e1d98715c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.009332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] The token cache is being invalidate because project 89c1bd62ecad464da702ab840d152b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:33.009645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.015532 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 976/1945] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:33.016764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.021834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90cb02-f4a7-4402-9397-8965fd5ace92 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:33.025730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 970/1946] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:32 2022] DELETE /identity/v3/projects/89c1bd62ecad464da702ab840d152b13 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:33.027979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:33.029655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:33.039103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6a90cb02-f4a7-4402-9397-8965fd5ace92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:33.152257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90cb02-f4a7-4402-9397-8965fd5ace92 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:33.203271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9143e86-20ca-42f0-8848-4b5e1d98715c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:33.223447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56279811-97c9-48d4-8123-c84f8fff6121 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:33.225443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56279811-97c9-48d4-8123-c84f8fff6121 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:33.225944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56279811-97c9-48d4-8123-c84f8fff6121 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:33.226414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56279811-97c9-48d4-8123-c84f8fff6121 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:33.240546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-56279811-97c9-48d4-8123-c84f8fff6121 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:33.249443 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:33.250128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:33.250394 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:33.250664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] PATH_INFO: `/v3/projects/7700613a4a4c447a830e3541b12b77e6` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:33.251935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7700613a4a4c447a830e3541b12b77e6)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:33.256720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:33.295114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:33.400330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7700613a4a4c447a830e3541b12b77e6', 'request_id': 'req-468a5b2c-fed1-4496-a4b4-beb29d09a11f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.419263 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7700613a4a4c447a830e3541b12b77e6', 'request_id': 'req-468a5b2c-fed1-4496-a4b4-beb29d09a11f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.428181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56279811-97c9-48d4-8123-c84f8fff6121 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:33.429751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56279811-97c9-48d4-8123-c84f8fff6121 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:33.431693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7700613a4a4c447a830e3541b12b77e6', 'request_id': 'req-468a5b2c-fed1-4496-a4b4-beb29d09a11f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.444322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 971/1947] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:33.445221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56279811-97c9-48d4-8123-c84f8fff6121 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:33.447491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-56279811-97c9-48d4-8123-c84f8fff6121 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:33.452241 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] The token cache is being invalidate because project 7700613a4a4c447a830e3541b12b77e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:33.452869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.454295 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.458312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 977/1948] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:33 2022] DELETE /identity/v3/projects/7700613a4a4c447a830e3541b12b77e6 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:33.459958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:33.463607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:33.510800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56279811-97c9-48d4-8123-c84f8fff6121 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:33.635341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a5b2c-fed1-4496-a4b4-beb29d09a11f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:33.637962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:33.639082 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:33.640538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:33.641043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] PATH_INFO: `/v3/users/7badc1c389f740c0afd7e2080600d20a` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:33.642725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55136dac-0981-497e-bd17-9595dbf10fda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:33.643160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7badc1c389f740c0afd7e2080600d20a)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:33.643397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55136dac-0981-497e-bd17-9595dbf10fda service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:33.644201 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55136dac-0981-497e-bd17-9595dbf10fda service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:33.648286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:33.651394 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55136dac-0981-497e-bd17-9595dbf10fda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:33.653584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-55136dac-0981-497e-bd17-9595dbf10fda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:33.730504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:33.756176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7badc1c389f740c0afd7e2080600d20a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.778854 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7badc1c389f740c0afd7e2080600d20a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.780584 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7badc1c389f740c0afd7e2080600d20a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.798662 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55136dac-0981-497e-bd17-9595dbf10fda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:33.800593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55136dac-0981-497e-bd17-9595dbf10fda service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:33.808267 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7badc1c389f740c0afd7e2080600d20a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.810643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7badc1c389f740c0afd7e2080600d20a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:33.826069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 978/1949] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:33.829462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55136dac-0981-497e-bd17-9595dbf10fda service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:33.835740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-55136dac-0981-497e-bd17-9595dbf10fda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:33.853495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 972/1950] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:33 2022] DELETE /identity/v3/users/7badc1c389f740c0afd7e2080600d20a => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:33.855989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:33.859036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:33.868751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55136dac-0981-497e-bd17-9595dbf10fda service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:34.075933 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e671ac-9d95-4a53-8cdf-950f6fe600e1 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:34.092398 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6473fdf-ada4-4826-96cc-5541f1c2ffe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:34.092398 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6473fdf-ada4-4826-96cc-5541f1c2ffe7 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:34.092816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6473fdf-ada4-4826-96cc-5541f1c2ffe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:34.093232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6473fdf-ada4-4826-96cc-5541f1c2ffe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:34.101140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b6473fdf-ada4-4826-96cc-5541f1c2ffe7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:34.114220 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:34.144971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe342c-d96a-4545-9e59-9a7c03f9f139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:34.150025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe342c-d96a-4545-9e59-9a7c03f9f139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:34.152040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe342c-d96a-4545-9e59-9a7c03f9f139 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:34.152945 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe342c-d96a-4545-9e59-9a7c03f9f139 admin admin] PATH_INFO: `/v3/users/75e906a711e148caa28a12a7dc4b9327` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:34.156135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe342c-d96a-4545-9e59-9a7c03f9f139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e906a711e148caa28a12a7dc4b9327)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:34.164226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe342c-d96a-4545-9e59-9a7c03f9f139 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:34.301705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6473fdf-ada4-4826-96cc-5541f1c2ffe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:34.303514 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6473fdf-ada4-4826-96cc-5541f1c2ffe7 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:34.328940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 973/1951] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:34.426205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-26fe342c-d96a-4545-9e59-9a7c03f9f139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e906a711e148caa28a12a7dc4b9327'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:34.483812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-26fe342c-d96a-4545-9e59-9a7c03f9f139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e906a711e148caa28a12a7dc4b9327'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:34.485674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-26fe342c-d96a-4545-9e59-9a7c03f9f139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e906a711e148caa28a12a7dc4b9327'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:34.489779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-26fe342c-d96a-4545-9e59-9a7c03f9f139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e906a711e148caa28a12a7dc4b9327'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:34.528971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-26fe342c-d96a-4545-9e59-9a7c03f9f139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e906a711e148caa28a12a7dc4b9327'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:34.547227 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6473fdf-ada4-4826-96cc-5541f1c2ffe7 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:34.551795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b6473fdf-ada4-4826-96cc-5541f1c2ffe7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:34.568641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 979/1952] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:33 2022] DELETE /identity/v3/users/75e906a711e148caa28a12a7dc4b9327 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:34.591935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6473fdf-ada4-4826-96cc-5541f1c2ffe7 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:34.900326 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b2819-54a0-46c8-af3d-f91470a43162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:34.901069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b2819-54a0-46c8-af3d-f91470a43162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:34.901463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b2819-54a0-46c8-af3d-f91470a43162 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:34.902500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b2819-54a0-46c8-af3d-f91470a43162 admin admin] PATH_INFO: `/v3/projects/ef8f9220e49d4510a60fea95048a59bb` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:34.918828 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b2819-54a0-46c8-af3d-f91470a43162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef8f9220e49d4510a60fea95048a59bb)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:34.923008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b2819-54a0-46c8-af3d-f91470a43162 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:34.943371 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:35.081530 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b91b2819-54a0-46c8-af3d-f91470a43162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef8f9220e49d4510a60fea95048a59bb', 'request_id': 'req-b91b2819-54a0-46c8-af3d-f91470a43162'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:35.093107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b91b2819-54a0-46c8-af3d-f91470a43162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef8f9220e49d4510a60fea95048a59bb', 'request_id': 'req-b91b2819-54a0-46c8-af3d-f91470a43162'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:35.110500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b91b2819-54a0-46c8-af3d-f91470a43162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef8f9220e49d4510a60fea95048a59bb', 'request_id': 'req-b91b2819-54a0-46c8-af3d-f91470a43162'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:35.160697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b91b2819-54a0-46c8-af3d-f91470a43162 admin admin] The token cache is being invalidate because project ef8f9220e49d4510a60fea95048a59bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:35.161000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b91b2819-54a0-46c8-af3d-f91470a43162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:35.166208 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b91b2819-54a0-46c8-af3d-f91470a43162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:35.179420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 974/1953] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:34 2022] DELETE /identity/v3/projects/ef8f9220e49d4510a60fea95048a59bb => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:35.184641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a5076-4ca9-4aa5-af61-85f1db0227e4 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:35.185258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a5076-4ca9-4aa5-af61-85f1db0227e4 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:35.185627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a5076-4ca9-4aa5-af61-85f1db0227e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:35.251598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-1d2a5076-4ca9-4aa5-af61-85f1db0227e4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:35.254308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1d2a5076-4ca9-4aa5-af61-85f1db0227e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:35.420216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 980/1954] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:35.435814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f661f-bc78-496d-ad0a-126df6373724 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:35.436667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f661f-bc78-496d-ad0a-126df6373724 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:35.436810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f661f-bc78-496d-ad0a-126df6373724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:35.475552 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-3e0f661f-bc78-496d-ad0a-126df6373724 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:35.476711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3e0f661f-bc78-496d-ad0a-126df6373724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:35.494847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 975/1955] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:35.500710 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a5076-4ca9-4aa5-af61-85f1db0227e4 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:35.521801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a5076-4ca9-4aa5-af61-85f1db0227e4 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:35.540438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf09d2-da86-4491-9a70-597cd7beb6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:35.543865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf09d2-da86-4491-9a70-597cd7beb6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:35.545641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf09d2-da86-4491-9a70-597cd7beb6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:35.545641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf09d2-da86-4491-9a70-597cd7beb6c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:35.556448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf09d2-da86-4491-9a70-597cd7beb6c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:35.558999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf09d2-da86-4491-9a70-597cd7beb6c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:35.564838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf09d2-da86-4491-9a70-597cd7beb6c7 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:35.608792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f661f-bc78-496d-ad0a-126df6373724 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:35.611152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3e0f661f-bc78-496d-ad0a-126df6373724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:35.641404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 981/1956] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:35.645370 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf09d2-da86-4491-9a70-597cd7beb6c7 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:35.681756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f661f-bc78-496d-ad0a-126df6373724 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:35.685806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf09d2-da86-4491-9a70-597cd7beb6c7 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:35.697130 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:35.698825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:35.699474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:35.700132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] PATH_INFO: `/v3/projects/63ab3819bdaa497ba6e65f03a701c636` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:35.702499 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63ab3819bdaa497ba6e65f03a701c636)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:35.703779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae53606-9d86-479d-a722-6d14b16aef16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:35.705778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae53606-9d86-479d-a722-6d14b16aef16 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:35.707331 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae53606-9d86-479d-a722-6d14b16aef16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:35.707551 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:35.710354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae53606-9d86-479d-a722-6d14b16aef16 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:35.711572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae53606-9d86-479d-a722-6d14b16aef16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:35.714540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae53606-9d86-479d-a722-6d14b16aef16 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:35.802380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 982/1957] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:35 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:35.807313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63ab3819bdaa497ba6e65f03a701c636', 'request_id': 'req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:35.809772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae53606-9d86-479d-a722-6d14b16aef16 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:35.815496 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63ab3819bdaa497ba6e65f03a701c636', 'request_id': 'req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:35.823675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63ab3819bdaa497ba6e65f03a701c636', 'request_id': 'req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:35.849141 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] The token cache is being invalidate because project 63ab3819bdaa497ba6e65f03a701c636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:35.851502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:35.854406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:35.856304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 976/1958] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:35 2022] DELETE /identity/v3/projects/63ab3819bdaa497ba6e65f03a701c636 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:35.862388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:35.867237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:35.915942 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:35.980270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae53606-9d86-479d-a722-6d14b16aef16 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:36.001959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec23eff-0173-4a3a-a3cc-3c625c518065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:36.003591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec23eff-0173-4a3a-a3cc-3c625c518065 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:36.005028 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec23eff-0173-4a3a-a3cc-3c625c518065 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:36.005523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec23eff-0173-4a3a-a3cc-3c625c518065 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:36.006732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec23eff-0173-4a3a-a3cc-3c625c518065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:36.010683 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec23eff-0173-4a3a-a3cc-3c625c518065 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:36.051240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 983/1959] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:35 2022] POST /identity/v3/users => generated 287 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:36.052168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3697bc1-9d60-4dce-a7ed-73cd0ca99bb2 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:36.055693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec23eff-0173-4a3a-a3cc-3c625c518065 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:36.066901 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2ec23eff-0173-4a3a-a3cc-3c625c518065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:36.073524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965abc55-36fc-44cc-b55e-2bde34919e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:36.076860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965abc55-36fc-44cc-b55e-2bde34919e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:36.077697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965abc55-36fc-44cc-b55e-2bde34919e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:36.078390 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965abc55-36fc-44cc-b55e-2bde34919e66 admin admin] PATH_INFO: `/v3/users/4785cd91963f4be8b9462f530d3c272b` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:36.087500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965abc55-36fc-44cc-b55e-2bde34919e66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4785cd91963f4be8b9462f530d3c272b)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:36.095098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965abc55-36fc-44cc-b55e-2bde34919e66 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:36.102584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec23eff-0173-4a3a-a3cc-3c625c518065 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:36.116363 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078c864-2e09-4aa0-984a-b65e0b371e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:36.117065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078c864-2e09-4aa0-984a-b65e0b371e00 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:36.117355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078c864-2e09-4aa0-984a-b65e0b371e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:36.117636 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078c864-2e09-4aa0-984a-b65e0b371e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:36.118439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078c864-2e09-4aa0-984a-b65e0b371e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:36.120548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078c864-2e09-4aa0-984a-b65e0b371e00 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:36.140300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 984/1960] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:36.141524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078c864-2e09-4aa0-984a-b65e0b371e00 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:36.151545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:36.159278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078c864-2e09-4aa0-984a-b65e0b371e00 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:36.166418 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a58f5-9cb0-40d7-be19-084614a2a276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:36.167082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a58f5-9cb0-40d7-be19-084614a2a276 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:36.167415 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a58f5-9cb0-40d7-be19-084614a2a276 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:36.167699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a58f5-9cb0-40d7-be19-084614a2a276 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:36.168401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a58f5-9cb0-40d7-be19-084614a2a276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:36.170157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a58f5-9cb0-40d7-be19-084614a2a276 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:36.201043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 985/1961] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:36.202762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a58f5-9cb0-40d7-be19-084614a2a276 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:36.230469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a58f5-9cb0-40d7-be19-084614a2a276 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:36.242674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-965abc55-36fc-44cc-b55e-2bde34919e66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4785cd91963f4be8b9462f530d3c272b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:36.244733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a6e6e-1ec2-49ac-a712-693a6696a5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:36.245442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a6e6e-1ec2-49ac-a712-693a6696a5c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:36.245759 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a6e6e-1ec2-49ac-a712-693a6696a5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:36.246809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a6e6e-1ec2-49ac-a712-693a6696a5c5 admin admin] PATH_INFO: `/v3/projects/03de73fafdda4985ae0bf5b7173a1482/users/5f340da546244ab7abe2c1987db701f9/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:36.270992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-965abc55-36fc-44cc-b55e-2bde34919e66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4785cd91963f4be8b9462f530d3c272b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:36.275400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-965abc55-36fc-44cc-b55e-2bde34919e66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4785cd91963f4be8b9462f530d3c272b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:36.317317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-965abc55-36fc-44cc-b55e-2bde34919e66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4785cd91963f4be8b9462f530d3c272b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:36.318654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-965abc55-36fc-44cc-b55e-2bde34919e66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4785cd91963f4be8b9462f530d3c272b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:36.320985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a6e6e-1ec2-49ac-a712-693a6696a5c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03de73fafdda4985ae0bf5b7173a1482, user_id=5f340da546244ab7abe2c1987db701f9, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:36.327514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a6e6e-1ec2-49ac-a712-693a6696a5c5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:36.358180 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 977/1962] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:35 2022] DELETE /identity/v3/users/4785cd91963f4be8b9462f530d3c272b => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:36.406369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 986/1963] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:36 2022] PUT /identity/v3/projects/03de73fafdda4985ae0bf5b7173a1482/users/5f340da546244ab7abe2c1987db701f9/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:36.434003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96494f24-82da-414a-aa73-23c3b4551950 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:36.434422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96494f24-82da-414a-aa73-23c3b4551950 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:36.434697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96494f24-82da-414a-aa73-23c3b4551950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:36.477525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-96494f24-82da-414a-aa73-23c3b4551950 None None] MFA Rules not processed for user `5f340da546244ab7abe2c1987db701f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:36.479138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-96494f24-82da-414a-aa73-23c3b4551950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:36.579605 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 978/1964] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:36.584534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a6e6e-1ec2-49ac-a712-693a6696a5c5 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:36.586480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7e9a6e6e-1ec2-49ac-a712-693a6696a5c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:36.720783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a6e6e-1ec2-49ac-a712-693a6696a5c5 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:36.728776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9600baa-5f95-4892-a5e9-f45e06a6b24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:36.729812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9600baa-5f95-4892-a5e9-f45e06a6b24b admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:36.730073 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9600baa-5f95-4892-a5e9-f45e06a6b24b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:36.730323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9600baa-5f95-4892-a5e9-f45e06a6b24b admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:36.731189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9600baa-5f95-4892-a5e9-f45e06a6b24b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:36.734554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9600baa-5f95-4892-a5e9-f45e06a6b24b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:36.772000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 987/1965] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:36 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:36.778230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96494f24-82da-414a-aa73-23c3b4551950 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:36.812316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96494f24-82da-414a-aa73-23c3b4551950 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:36.940478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91f0d1-0da0-4bbd-aad3-a7c1b23dada4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:36.941676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91f0d1-0da0-4bbd-aad3-a7c1b23dada4 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:36.941956 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91f0d1-0da0-4bbd-aad3-a7c1b23dada4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:36.942392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91f0d1-0da0-4bbd-aad3-a7c1b23dada4 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:36.943198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91f0d1-0da0-4bbd-aad3-a7c1b23dada4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:36.945505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91f0d1-0da0-4bbd-aad3-a7c1b23dada4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:36.976115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9600baa-5f95-4892-a5e9-f45e06a6b24b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:36.980253 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e9600baa-5f95-4892-a5e9-f45e06a6b24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:36.983532 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 979/1966] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:36 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:36.987479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91f0d1-0da0-4bbd-aad3-a7c1b23dada4 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:37.002274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91f0d1-0da0-4bbd-aad3-a7c1b23dada4 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:37.008524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392ffc3-b7e7-4315-bb8b-79edd924203f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:37.009303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392ffc3-b7e7-4315-bb8b-79edd924203f admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.009758 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392ffc3-b7e7-4315-bb8b-79edd924203f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.010461 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9600baa-5f95-4892-a5e9-f45e06a6b24b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:37.010639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392ffc3-b7e7-4315-bb8b-79edd924203f admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.012136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392ffc3-b7e7-4315-bb8b-79edd924203f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:37.015456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392ffc3-b7e7-4315-bb8b-79edd924203f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:37.036167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:37.037348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.037742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.038238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc admin admin] PATH_INFO: `/v3/projects/032e879fb7e24676973d6a08c44b6d58` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.044999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=032e879fb7e24676973d6a08c44b6d58)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:37.049919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:37.055045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 980/1967] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:37.061135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392ffc3-b7e7-4315-bb8b-79edd924203f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:37.079872 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:37.093998 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392ffc3-b7e7-4315-bb8b-79edd924203f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:37.105492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a582b-e476-4307-a7e1-6e9fc0d395e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:37.106266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a582b-e476-4307-a7e1-6e9fc0d395e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.106762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a582b-e476-4307-a7e1-6e9fc0d395e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.107117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a582b-e476-4307-a7e1-6e9fc0d395e0 admin admin] PATH_INFO: `/v3/projects/216fac093d094f708e659f5927204abb/users/14053590184f4e7dbbef122b3822fefe/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.136601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a582b-e476-4307-a7e1-6e9fc0d395e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=216fac093d094f708e659f5927204abb, user_id=14053590184f4e7dbbef122b3822fefe, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:37.141233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a582b-e476-4307-a7e1-6e9fc0d395e0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:37.162712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 981/1968] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:37 2022] PUT /identity/v3/projects/216fac093d094f708e659f5927204abb/users/14053590184f4e7dbbef122b3822fefe/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:37.167172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a582b-e476-4307-a7e1-6e9fc0d395e0 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:37.182658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '032e879fb7e24676973d6a08c44b6d58', 'request_id': 'req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.191953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '032e879fb7e24676973d6a08c44b6d58', 'request_id': 'req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.204134 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '032e879fb7e24676973d6a08c44b6d58', 'request_id': 'req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.206657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:37.225164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc admin admin] The token cache is being invalidate because project 032e879fb7e24676973d6a08c44b6d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:37.225716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.228314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f2cd91f-1a1f-4198-a28d-1cf829ebe1fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.235469 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 988/1969] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:36 2022] DELETE /identity/v3/projects/032e879fb7e24676973d6a08c44b6d58 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:37.239944 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9bff7-f67e-49b7-af0d-f4c79af43f15 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.240138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9bff7-f67e-49b7-af0d-f4c79af43f15 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.240359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9bff7-f67e-49b7-af0d-f4c79af43f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.268113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-d2d9bff7-f67e-49b7-af0d-f4c79af43f15 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:37.269805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d2d9bff7-f67e-49b7-af0d-f4c79af43f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:37.296738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a582b-e476-4307-a7e1-6e9fc0d395e0 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:37.318988 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a3843-0902-4a01-9322-cb5d3d9209c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:37.319663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a3843-0902-4a01-9322-cb5d3d9209c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.319946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a3843-0902-4a01-9322-cb5d3d9209c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.320216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a3843-0902-4a01-9322-cb5d3d9209c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.320875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a3843-0902-4a01-9322-cb5d3d9209c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:37.322538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a3843-0902-4a01-9322-cb5d3d9209c9 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:37.365863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 982/1970] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:37.369417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a3843-0902-4a01-9322-cb5d3d9209c9 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:37.373967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-710a3843-0902-4a01-9322-cb5d3d9209c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:37.387004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 989/1971] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:37.398140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3048e-8d40-4c94-85d5-660ba9167ca3 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.398421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3048e-8d40-4c94-85d5-660ba9167ca3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.398699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3048e-8d40-4c94-85d5-660ba9167ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.411380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a3843-0902-4a01-9322-cb5d3d9209c9 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:37.433915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-4cb3048e-8d40-4c94-85d5-660ba9167ca3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:37.435279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4cb3048e-8d40-4c94-85d5-660ba9167ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:37.436583 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3034f34-b2cc-43b5-ae64-8abaa3283032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:37.437788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3034f34-b2cc-43b5-ae64-8abaa3283032 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.438314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3034f34-b2cc-43b5-ae64-8abaa3283032 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.438680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3034f34-b2cc-43b5-ae64-8abaa3283032 admin admin] PATH_INFO: `/v3/projects/216fac093d094f708e659f5927204abb/users/14053590184f4e7dbbef122b3822fefe/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.445630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3034f34-b2cc-43b5-ae64-8abaa3283032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=216fac093d094f708e659f5927204abb, user_id=14053590184f4e7dbbef122b3822fefe, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:37.453629 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 990/1972] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:37.455071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3048e-8d40-4c94-85d5-660ba9167ca3 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:37.460787 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3034f34-b2cc-43b5-ae64-8abaa3283032 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:37.473083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3048e-8d40-4c94-85d5-660ba9167ca3 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:37.507293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 983/1973] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:37 2022] PUT /identity/v3/projects/216fac093d094f708e659f5927204abb/users/14053590184f4e7dbbef122b3822fefe/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:37.518921 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee825e-9dce-4f6d-b1ea-9acc098f79b6 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.519659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee825e-9dce-4f6d-b1ea-9acc098f79b6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.519659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee825e-9dce-4f6d-b1ea-9acc098f79b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.573107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-93ee825e-9dce-4f6d-b1ea-9acc098f79b6 None None] MFA Rules not processed for user `14053590184f4e7dbbef122b3822fefe`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:37.574405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-93ee825e-9dce-4f6d-b1ea-9acc098f79b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:37.601095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27809-3852-4996-ba72-8c9af80303e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:37.602174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27809-3852-4996-ba72-8c9af80303e1 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.602641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27809-3852-4996-ba72-8c9af80303e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.603183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27809-3852-4996-ba72-8c9af80303e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.604689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc27809-3852-4996-ba72-8c9af80303e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:37.606057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc27809-3852-4996-ba72-8c9af80303e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:37.610399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc27809-3852-4996-ba72-8c9af80303e1 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:37.646915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 991/1974] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:37.648798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27809-3852-4996-ba72-8c9af80303e1 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:37.651980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-edc27809-3852-4996-ba72-8c9af80303e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:37.693847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27809-3852-4996-ba72-8c9af80303e1 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:37.709844 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:37.711609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.712368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.713121 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] PATH_INFO: `/v3/users/67cce6bbc10c4b199dcc199b87c4c0cf` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.719418 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67cce6bbc10c4b199dcc199b87c4c0cf)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:37.730009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:37.754620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 984/1975] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:37.758739 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee825e-9dce-4f6d-b1ea-9acc098f79b6 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:37.774238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee825e-9dce-4f6d-b1ea-9acc098f79b6 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:37.779886 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c769a9-8104-4e21-8b64-cfbc1cadf7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:37.781802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c769a9-8104-4e21-8b64-cfbc1cadf7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.783229 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c769a9-8104-4e21-8b64-cfbc1cadf7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.783800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c769a9-8104-4e21-8b64-cfbc1cadf7a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.786005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c769a9-8104-4e21-8b64-cfbc1cadf7a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:37.791783 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c769a9-8104-4e21-8b64-cfbc1cadf7a1 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:37.845146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67cce6bbc10c4b199dcc199b87c4c0cf'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.872659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 985/1976] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:37 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:37.873794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c769a9-8104-4e21-8b64-cfbc1cadf7a1 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:37.877442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-94c769a9-8104-4e21-8b64-cfbc1cadf7a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:37.893588 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67cce6bbc10c4b199dcc199b87c4c0cf'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.895850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67cce6bbc10c4b199dcc199b87c4c0cf'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.898785 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67cce6bbc10c4b199dcc199b87c4c0cf'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.923762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67cce6bbc10c4b199dcc199b87c4c0cf'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.926805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:37.949245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 992/1977] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:37 2022] DELETE /identity/v3/users/67cce6bbc10c4b199dcc199b87c4c0cf => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:37.951921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:37.955509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:37.982153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:38.072593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c769a9-8104-4e21-8b64-cfbc1cadf7a1 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:38.124390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe33a58-af81-4c1d-87be-e237f40f8a56 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:38.137202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0d7e8-127a-4263-8e83-5d3797887f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:38.138878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0d7e8-127a-4263-8e83-5d3797887f38 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:38.139551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0d7e8-127a-4263-8e83-5d3797887f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:38.140165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0d7e8-127a-4263-8e83-5d3797887f38 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:38.142236 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0d7e8-127a-4263-8e83-5d3797887f38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:38.147043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0d7e8-127a-4263-8e83-5d3797887f38 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:38.168975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9a36d-dded-463d-aaa3-1ae33e95cc57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:38.170174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9a36d-dded-463d-aaa3-1ae33e95cc57 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:38.170691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9a36d-dded-463d-aaa3-1ae33e95cc57 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:38.170942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9a36d-dded-463d-aaa3-1ae33e95cc57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:38.174316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4ca9a36d-dded-463d-aaa3-1ae33e95cc57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:38.180825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 993/1978] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:37 2022] POST /identity/v3/users => generated 291 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:38.186594 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1cf09-790f-4187-b5b5-cd892968230c None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:38.186953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1cf09-790f-4187-b5b5-cd892968230c None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:38.187348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1cf09-790f-4187-b5b5-cd892968230c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:38.224789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-caf1cf09-790f-4187-b5b5-cd892968230c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:38.226102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-caf1cf09-790f-4187-b5b5-cd892968230c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:38.260545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 994/1979] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:38.261846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1cf09-790f-4187-b5b5-cd892968230c None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:38.276656 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1cf09-790f-4187-b5b5-cd892968230c None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:38.284357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e8032-125c-42f6-a785-379a61014b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:38.285139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e8032-125c-42f6-a785-379a61014b94 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:38.285424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e8032-125c-42f6-a785-379a61014b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:38.285701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e8032-125c-42f6-a785-379a61014b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:38.286373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e8032-125c-42f6-a785-379a61014b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:38.288439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e8032-125c-42f6-a785-379a61014b94 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:38.312925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 995/1980] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:38.315025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e8032-125c-42f6-a785-379a61014b94 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:38.317927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a23e8032-125c-42f6-a785-379a61014b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:38.355714 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e8032-125c-42f6-a785-379a61014b94 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:38.362695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9a36d-dded-463d-aaa3-1ae33e95cc57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:38.365498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9a36d-dded-463d-aaa3-1ae33e95cc57 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:38.392886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24553f99-cc16-481b-9c4a-b827a6c7d112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:38.394005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 986/1981] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:38.394190 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24553f99-cc16-481b-9c4a-b827a6c7d112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:38.394762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9a36d-dded-463d-aaa3-1ae33e95cc57 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:38.394932 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24553f99-cc16-481b-9c4a-b827a6c7d112 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:38.395555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24553f99-cc16-481b-9c4a-b827a6c7d112 admin admin] PATH_INFO: `/v3/users/cbfd91621f3340c2a71ce61ba9e0532a` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:38.400723 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24553f99-cc16-481b-9c4a-b827a6c7d112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbfd91621f3340c2a71ce61ba9e0532a)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:38.409292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9a36d-dded-463d-aaa3-1ae33e95cc57 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:38.412294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24553f99-cc16-481b-9c4a-b827a6c7d112 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:38.414739 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6a54c-3bd4-4db5-8b47-156641f83e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:38.415453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6a54c-3bd4-4db5-8b47-156641f83e07 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:38.415735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6a54c-3bd4-4db5-8b47-156641f83e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:38.416012 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6a54c-3bd4-4db5-8b47-156641f83e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:38.416680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6a54c-3bd4-4db5-8b47-156641f83e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:38.419250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6a54c-3bd4-4db5-8b47-156641f83e07 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:38.434749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 987/1982] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:38.440396 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6a54c-3bd4-4db5-8b47-156641f83e07 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:38.443980 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3fc6a54c-3bd4-4db5-8b47-156641f83e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:38.582300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-24553f99-cc16-481b-9c4a-b827a6c7d112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbfd91621f3340c2a71ce61ba9e0532a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.596410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6a54c-3bd4-4db5-8b47-156641f83e07 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:38.608598 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3eeebb-2fb6-4576-980e-5cb0e0f07f47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:38.609675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3eeebb-2fb6-4576-980e-5cb0e0f07f47 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:38.610165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3eeebb-2fb6-4576-980e-5cb0e0f07f47 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:38.610688 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3eeebb-2fb6-4576-980e-5cb0e0f07f47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:38.633756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3eeebb-2fb6-4576-980e-5cb0e0f07f47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:38.637159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3eeebb-2fb6-4576-980e-5cb0e0f07f47 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:38.638720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-24553f99-cc16-481b-9c4a-b827a6c7d112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbfd91621f3340c2a71ce61ba9e0532a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.640899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-24553f99-cc16-481b-9c4a-b827a6c7d112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbfd91621f3340c2a71ce61ba9e0532a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.644956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-24553f99-cc16-481b-9c4a-b827a6c7d112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbfd91621f3340c2a71ce61ba9e0532a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.659063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 988/1983] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:38.661149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3eeebb-2fb6-4576-980e-5cb0e0f07f47 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:38.674038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-24553f99-cc16-481b-9c4a-b827a6c7d112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbfd91621f3340c2a71ce61ba9e0532a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.679890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3eeebb-2fb6-4576-980e-5cb0e0f07f47 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:38.698635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42c0f8-83a7-4110-a21b-f9af9bdf0810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:38.699653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42c0f8-83a7-4110-a21b-f9af9bdf0810 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:38.700127 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42c0f8-83a7-4110-a21b-f9af9bdf0810 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:38.700718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42c0f8-83a7-4110-a21b-f9af9bdf0810 admin admin] PATH_INFO: `/v3/projects/79f580f598c8478a964c189dd441a841/users/592a5d46fd81440d92f9167bf6002831/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:38.741032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 996/1984] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:38 2022] DELETE /identity/v3/users/cbfd91621f3340c2a71ce61ba9e0532a => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:38.753209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff4613-f966-4175-8316-5d0a2d01a242 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:38.755168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff4613-f966-4175-8316-5d0a2d01a242 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:38.755507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff4613-f966-4175-8316-5d0a2d01a242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:38.762102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42c0f8-83a7-4110-a21b-f9af9bdf0810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f580f598c8478a964c189dd441a841, user_id=592a5d46fd81440d92f9167bf6002831, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:38.773169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42c0f8-83a7-4110-a21b-f9af9bdf0810 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:38.820422 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-3dff4613-f966-4175-8316-5d0a2d01a242 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:38.821987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3dff4613-f966-4175-8316-5d0a2d01a242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:38.836802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 989/1985] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:38 2022] PUT /identity/v3/projects/79f580f598c8478a964c189dd441a841/users/592a5d46fd81440d92f9167bf6002831/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:38.848386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41344ca7-e92a-42ae-b094-b16d89a548b8 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:38.858488 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41344ca7-e92a-42ae-b094-b16d89a548b8 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:38.859576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41344ca7-e92a-42ae-b094-b16d89a548b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:38.893851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:38.908441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:38.956321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-41344ca7-e92a-42ae-b094-b16d89a548b8 None None] MFA Rules not processed for user `592a5d46fd81440d92f9167bf6002831`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:38.959676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-41344ca7-e92a-42ae-b094-b16d89a548b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:39.007810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 997/1986] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:39.092878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff4613-f966-4175-8316-5d0a2d01a242 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.095478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3dff4613-f966-4175-8316-5d0a2d01a242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:39.170127 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 990/1987] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:39.174325 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41344ca7-e92a-42ae-b094-b16d89a548b8 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.176637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-41344ca7-e92a-42ae-b094-b16d89a548b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:39.201129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41344ca7-e92a-42ae-b094-b16d89a548b8 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.208442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1f00b-b5d3-4b0f-b0d3-c99bdd29a242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.210431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1f00b-b5d3-4b0f-b0d3-c99bdd29a242 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.210711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1f00b-b5d3-4b0f-b0d3-c99bdd29a242 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.210982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1f00b-b5d3-4b0f-b0d3-c99bdd29a242 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.227165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1f00b-b5d3-4b0f-b0d3-c99bdd29a242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.227165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1f00b-b5d3-4b0f-b0d3-c99bdd29a242 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.231834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff4613-f966-4175-8316-5d0a2d01a242 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.238943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d4b79-8ae9-49be-a45a-71354d0f66d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.240150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d4b79-8ae9-49be-a45a-71354d0f66d5 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.240485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d4b79-8ae9-49be-a45a-71354d0f66d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.240711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d4b79-8ae9-49be-a45a-71354d0f66d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.268067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 991/1988] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:39 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:39.268905 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d4b79-8ae9-49be-a45a-71354d0f66d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.272008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1f00b-b5d3-4b0f-b0d3-c99bdd29a242 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.272880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d4b79-8ae9-49be-a45a-71354d0f66d5 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.296756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 998/1989] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:39.362096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1f00b-b5d3-4b0f-b0d3-c99bdd29a242 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.367958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2f52a-0d73-42f6-aabd-071e813969e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.368594 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2f52a-0d73-42f6-aabd-071e813969e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.368872 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2f52a-0d73-42f6-aabd-071e813969e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.369142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2f52a-0d73-42f6-aabd-071e813969e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.369800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2f52a-0d73-42f6-aabd-071e813969e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.371870 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2f52a-0d73-42f6-aabd-071e813969e5 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.398554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 992/1990] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:39 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:39.402724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d4b79-8ae9-49be-a45a-71354d0f66d5 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.414667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d4b79-8ae9-49be-a45a-71354d0f66d5 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.419721 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda37814-ecbc-4f5a-8b24-347b9d7c571b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.420507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda37814-ecbc-4f5a-8b24-347b9d7c571b admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.420835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda37814-ecbc-4f5a-8b24-347b9d7c571b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.421121 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda37814-ecbc-4f5a-8b24-347b9d7c571b admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.421774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda37814-ecbc-4f5a-8b24-347b9d7c571b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.423700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda37814-ecbc-4f5a-8b24-347b9d7c571b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.436744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 999/1991] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:39.443840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2f52a-0d73-42f6-aabd-071e813969e5 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.458118 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2f52a-0d73-42f6-aabd-071e813969e5 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.463330 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc17f8-bb22-48bb-b3b5-9ade51d0d77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.464019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc17f8-bb22-48bb-b3b5-9ade51d0d77e admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.464300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc17f8-bb22-48bb-b3b5-9ade51d0d77e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.464576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc17f8-bb22-48bb-b3b5-9ade51d0d77e admin admin] PATH_INFO: `/v3/projects/1452591191f94e55b9c4a313d9dee8c0/users/2ac12a69255a4c4fa00f9ad35e248950/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.483502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc17f8-bb22-48bb-b3b5-9ade51d0d77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1452591191f94e55b9c4a313d9dee8c0, user_id=2ac12a69255a4c4fa00f9ad35e248950, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.486601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc17f8-bb22-48bb-b3b5-9ade51d0d77e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.504953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 993/1992] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:39 2022] PUT /identity/v3/projects/1452591191f94e55b9c4a313d9dee8c0/users/2ac12a69255a4c4fa00f9ad35e248950/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:39.509493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda37814-ecbc-4f5a-8b24-347b9d7c571b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.521347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda37814-ecbc-4f5a-8b24-347b9d7c571b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.594719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8564d5-6dc1-4eac-a793-b9253b22a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.595435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8564d5-6dc1-4eac-a793-b9253b22a5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.595714 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8564d5-6dc1-4eac-a793-b9253b22a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.595993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8564d5-6dc1-4eac-a793-b9253b22a5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.596648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8564d5-6dc1-4eac-a793-b9253b22a5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.599256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8564d5-6dc1-4eac-a793-b9253b22a5b4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.617465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1000/1993] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:39.619752 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc17f8-bb22-48bb-b3b5-9ade51d0d77e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.633467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc17f8-bb22-48bb-b3b5-9ade51d0d77e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.640866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57e062-0e0c-4cf8-a1ca-16943b278fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.643409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57e062-0e0c-4cf8-a1ca-16943b278fef admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.643806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57e062-0e0c-4cf8-a1ca-16943b278fef admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.644755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57e062-0e0c-4cf8-a1ca-16943b278fef admin admin] PATH_INFO: `/v3/projects/1452591191f94e55b9c4a313d9dee8c0/users/2ac12a69255a4c4fa00f9ad35e248950/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.648749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57e062-0e0c-4cf8-a1ca-16943b278fef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1452591191f94e55b9c4a313d9dee8c0, user_id=2ac12a69255a4c4fa00f9ad35e248950, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.651411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57e062-0e0c-4cf8-a1ca-16943b278fef admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.667919 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 994/1994] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:39 2022] PUT /identity/v3/projects/1452591191f94e55b9c4a313d9dee8c0/users/2ac12a69255a4c4fa00f9ad35e248950/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:39.680818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82d9da-d473-4f7e-9d09-c88b90b95dcd None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.681104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82d9da-d473-4f7e-9d09-c88b90b95dcd None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.681407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82d9da-d473-4f7e-9d09-c88b90b95dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.714906 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0a82d9da-d473-4f7e-9d09-c88b90b95dcd None None] MFA Rules not processed for user `2ac12a69255a4c4fa00f9ad35e248950`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:39.716158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0a82d9da-d473-4f7e-9d09-c88b90b95dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:39.789304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:39.820323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1001/1995] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:39.828415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57e062-0e0c-4cf8-a1ca-16943b278fef admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.830769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1f57e062-0e0c-4cf8-a1ca-16943b278fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:39.861951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:39.903295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57e062-0e0c-4cf8-a1ca-16943b278fef admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.908978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bda0c8-d249-48f4-8966-ed1b455ec858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.909781 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bda0c8-d249-48f4-8966-ed1b455ec858 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.910254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bda0c8-d249-48f4-8966-ed1b455ec858 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.911108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bda0c8-d249-48f4-8966-ed1b455ec858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.923385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bda0c8-d249-48f4-8966-ed1b455ec858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.925117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bda0c8-d249-48f4-8966-ed1b455ec858 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.937839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 995/1996] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:40.070619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82d9da-d473-4f7e-9d09-c88b90b95dcd None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:40.094556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82d9da-d473-4f7e-9d09-c88b90b95dcd None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:40.101936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bda0c8-d249-48f4-8966-ed1b455ec858 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:40.103587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-56bda0c8-d249-48f4-8966-ed1b455ec858 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:40.111081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9340c06-0234-446f-99fa-159ed2824773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:40.111931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9340c06-0234-446f-99fa-159ed2824773 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:40.112144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9340c06-0234-446f-99fa-159ed2824773 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:40.112580 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9340c06-0234-446f-99fa-159ed2824773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:40.230049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9340c06-0234-446f-99fa-159ed2824773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:40.232685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9340c06-0234-446f-99fa-159ed2824773 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:40.253182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1002/1997] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:40.274046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bda0c8-d249-48f4-8966-ed1b455ec858 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:40.286135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:40.287531 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:40.288112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:40.288683 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] PATH_INFO: `/v3/projects/2c3e3d29f9cc46afbf7b6800a1ebe05a` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:40.290759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c3e3d29f9cc46afbf7b6800a1ebe05a)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:40.294793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:40.404909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c3e3d29f9cc46afbf7b6800a1ebe05a', 'request_id': 'req-b4098183-7e88-4401-9740-82381ac930aa'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:40.414776 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c3e3d29f9cc46afbf7b6800a1ebe05a', 'request_id': 'req-b4098183-7e88-4401-9740-82381ac930aa'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:40.438643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c3e3d29f9cc46afbf7b6800a1ebe05a', 'request_id': 'req-b4098183-7e88-4401-9740-82381ac930aa'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:40.480370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] The token cache is being invalidate because project 2c3e3d29f9cc46afbf7b6800a1ebe05a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:40.480917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:40.482562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:40.484650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 996/1998] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:40 2022] DELETE /identity/v3/projects/2c3e3d29f9cc46afbf7b6800a1ebe05a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:40.489742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9340c06-0234-446f-99fa-159ed2824773 service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:40.504839 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9340c06-0234-446f-99fa-159ed2824773 service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:40.661953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:40.662657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:40.663060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:40.663371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] PATH_INFO: `/v3/users/4734c778339b46879925b6c2adbbbc08` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:40.665084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4734c778339b46879925b6c2adbbbc08)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:40.667322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:40.744980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4734c778339b46879925b6c2adbbbc08'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:40.780120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4734c778339b46879925b6c2adbbbc08'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:40.780861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4734c778339b46879925b6c2adbbbc08'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:40.781733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4734c778339b46879925b6c2adbbbc08'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:40.793386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4734c778339b46879925b6c2adbbbc08'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:40.820395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1003/1999] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:40 2022] DELETE /identity/v3/users/4734c778339b46879925b6c2adbbbc08 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:41.016791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:41.018541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:41.055646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4098183-7e88-4401-9740-82381ac930aa admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:41.142515 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:41.152777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:41.153493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:41.154170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:41.154764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] PATH_INFO: `/v3/projects/87fdd1483c9140e692115d0ad7b14780` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:41.156754 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] RBAC: Authorizing `identity:delete_project(project_id=87fdd1483c9140e692115d0ad7b14780)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:41.160169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:41.246419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87fdd1483c9140e692115d0ad7b14780', 'request_id': 'req-d44ecda1-20db-4230-807a-f273fdbe8ece'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.261248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87fdd1483c9140e692115d0ad7b14780', 'request_id': 'req-d44ecda1-20db-4230-807a-f273fdbe8ece'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.271314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87fdd1483c9140e692115d0ad7b14780', 'request_id': 'req-d44ecda1-20db-4230-807a-f273fdbe8ece'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.301945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] The token cache is being invalidate because project 87fdd1483c9140e692115d0ad7b14780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:41.302879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.303309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.310813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 997/2000] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:41 2022] DELETE /identity/v3/projects/87fdd1483c9140e692115d0ad7b14780 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:41.312450 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:41.314796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:41.346513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:41.355062 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baebc3d-25c8-4e8f-934f-443002aac9bb admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:41.414243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:41.416662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:41.489222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:41.489922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:41.490212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:41.490475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] PATH_INFO: `/v3/users/e9e0eefa163844aa8a98b19ab8d5c6c4` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:41.491973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9e0eefa163844aa8a98b19ab8d5c6c4)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:41.494387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:41.540584 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ecda1-20db-4230-807a-f273fdbe8ece admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:41.549067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abec571-8256-4438-b770-f52729f156a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:41.549388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abec571-8256-4438-b770-f52729f156a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:41.549677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abec571-8256-4438-b770-f52729f156a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:41.550193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abec571-8256-4438-b770-f52729f156a0 admin admin] PATH_INFO: `/v3/projects/63fea73a2d9a4735843579be6b8b0ec4` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:41.552947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abec571-8256-4438-b770-f52729f156a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63fea73a2d9a4735843579be6b8b0ec4)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:41.556068 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abec571-8256-4438-b770-f52729f156a0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:41.595451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9e0eefa163844aa8a98b19ab8d5c6c4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.636118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9e0eefa163844aa8a98b19ab8d5c6c4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.637478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3abec571-8256-4438-b770-f52729f156a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63fea73a2d9a4735843579be6b8b0ec4', 'request_id': 'req-3abec571-8256-4438-b770-f52729f156a0'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.637888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9e0eefa163844aa8a98b19ab8d5c6c4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.639600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9e0eefa163844aa8a98b19ab8d5c6c4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.652751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3abec571-8256-4438-b770-f52729f156a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63fea73a2d9a4735843579be6b8b0ec4', 'request_id': 'req-3abec571-8256-4438-b770-f52729f156a0'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.656446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9e0eefa163844aa8a98b19ab8d5c6c4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.688324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1004/2001] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:41 2022] DELETE /identity/v3/users/e9e0eefa163844aa8a98b19ab8d5c6c4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:41.693725 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3abec571-8256-4438-b770-f52729f156a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63fea73a2d9a4735843579be6b8b0ec4', 'request_id': 'req-3abec571-8256-4438-b770-f52729f156a0'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.704249 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:41.706487 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:41.730623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3abec571-8256-4438-b770-f52729f156a0 admin admin] The token cache is being invalidate because project 63fea73a2d9a4735843579be6b8b0ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:41.730968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3abec571-8256-4438-b770-f52729f156a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.733739 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3abec571-8256-4438-b770-f52729f156a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:41.735866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 998/2002] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:41 2022] DELETE /identity/v3/projects/63fea73a2d9a4735843579be6b8b0ec4 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:41.741534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4889a79-be0f-43fe-9010-a5f14b2b5530 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:41.742038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4889a79-be0f-43fe-9010-a5f14b2b5530 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:41.743165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4889a79-be0f-43fe-9010-a5f14b2b5530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:41.789389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f4889a79-be0f-43fe-9010-a5f14b2b5530 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:41.792812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f4889a79-be0f-43fe-9010-a5f14b2b5530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:41.817811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:41.852800 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef75f25-87d4-4005-b764-12b50ff458ed admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:41.867033 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a636f-f57a-4d50-affd-76681cda99c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:41.870118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a636f-f57a-4d50-affd-76681cda99c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:41.870525 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a636f-f57a-4d50-affd-76681cda99c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:41.870932 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a636f-f57a-4d50-affd-76681cda99c2 admin admin] PATH_INFO: `/v3/users/5f340da546244ab7abe2c1987db701f9` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:41.873638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a636f-f57a-4d50-affd-76681cda99c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f340da546244ab7abe2c1987db701f9)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:41.877892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a636f-f57a-4d50-affd-76681cda99c2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:41.937856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:41.949106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 999/2003] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:41.962067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d85fa-28ad-4d39-9064-052fb5ee92ec None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:41.962067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d85fa-28ad-4d39-9064-052fb5ee92ec None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:41.962067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d85fa-28ad-4d39-9064-052fb5ee92ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:41.983419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a35a636f-f57a-4d50-affd-76681cda99c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f340da546244ab7abe2c1987db701f9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:42.047097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a35a636f-f57a-4d50-affd-76681cda99c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f340da546244ab7abe2c1987db701f9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:42.050309 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a35a636f-f57a-4d50-affd-76681cda99c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f340da546244ab7abe2c1987db701f9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:42.052341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a35a636f-f57a-4d50-affd-76681cda99c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f340da546244ab7abe2c1987db701f9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:42.055581 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-080d85fa-28ad-4d39-9064-052fb5ee92ec None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:42.057663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-080d85fa-28ad-4d39-9064-052fb5ee92ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:42.067539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a35a636f-f57a-4d50-affd-76681cda99c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f340da546244ab7abe2c1987db701f9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:42.086491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1000/2004] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:42.094405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d85fa-28ad-4d39-9064-052fb5ee92ec None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:42.102762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1005/2005] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:41 2022] DELETE /identity/v3/users/5f340da546244ab7abe2c1987db701f9 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:42.108523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9386-c019-474a-ac16-ee69676c8e7c None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:42.110497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9386-c019-474a-ac16-ee69676c8e7c None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:42.110865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9386-c019-474a-ac16-ee69676c8e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:42.118204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d85fa-28ad-4d39-9064-052fb5ee92ec None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:42.177455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-25fe9386-c019-474a-ac16-ee69676c8e7c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:42.178755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-25fe9386-c019-474a-ac16-ee69676c8e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:42.214847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b894f76-80e5-4c8a-91d6-9ff11ff272a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:42.216301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b894f76-80e5-4c8a-91d6-9ff11ff272a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:42.217803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b894f76-80e5-4c8a-91d6-9ff11ff272a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:42.218173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b894f76-80e5-4c8a-91d6-9ff11ff272a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:42.219682 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b894f76-80e5-4c8a-91d6-9ff11ff272a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:42.220017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b894f76-80e5-4c8a-91d6-9ff11ff272a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:42.221662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b894f76-80e5-4c8a-91d6-9ff11ff272a6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:42.247273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1001/2006] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:42.250361 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b894f76-80e5-4c8a-91d6-9ff11ff272a6 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:42.265773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b894f76-80e5-4c8a-91d6-9ff11ff272a6 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:42.291909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38f922-bf59-45e3-a621-8786229ca2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:42.292877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38f922-bf59-45e3-a621-8786229ca2ad admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:42.293763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38f922-bf59-45e3-a621-8786229ca2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:42.293763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38f922-bf59-45e3-a621-8786229ca2ad admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:42.294266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38f922-bf59-45e3-a621-8786229ca2ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:42.296281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38f922-bf59-45e3-a621-8786229ca2ad admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:42.313541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1006/2007] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:42.324608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe9386-c019-474a-ac16-ee69676c8e7c None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:42.326220 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-25fe9386-c019-474a-ac16-ee69676c8e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:42.354904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1002/2008] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:42 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:42.359150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38f922-bf59-45e3-a621-8786229ca2ad admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:42.437895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe9386-c019-474a-ac16-ee69676c8e7c None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:42.492627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38f922-bf59-45e3-a621-8786229ca2ad admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:42.499198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9ce9e-7f23-4a8b-b346-a9b3cff5f8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:42.499841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9ce9e-7f23-4a8b-b346-a9b3cff5f8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:42.500107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9ce9e-7f23-4a8b-b346-a9b3cff5f8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:42.500427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9ce9e-7f23-4a8b-b346-a9b3cff5f8f2 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:42.501798 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9ce9e-7f23-4a8b-b346-a9b3cff5f8f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:42.504682 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9ce9e-7f23-4a8b-b346-a9b3cff5f8f2 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:42.544245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1003/2009] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:42 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:42.546288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9ce9e-7f23-4a8b-b346-a9b3cff5f8f2 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:42.548925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-04d9ce9e-7f23-4a8b-b346-a9b3cff5f8f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:42.555293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d200e11-b37a-4e76-9dc5-1d2dd901301f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:42.556080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d200e11-b37a-4e76-9dc5-1d2dd901301f service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:42.556486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d200e11-b37a-4e76-9dc5-1d2dd901301f service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:42.556865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d200e11-b37a-4e76-9dc5-1d2dd901301f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:42.567879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9ce9e-7f23-4a8b-b346-a9b3cff5f8f2 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:42.571080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d200e11-b37a-4e76-9dc5-1d2dd901301f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:42.572919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d200e11-b37a-4e76-9dc5-1d2dd901301f service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:42.574965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:42.575815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:42.576098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:42.576369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] PATH_INFO: `/v3/projects/ab884a4da54646aa9079d820192f21b1` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:42.578639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab884a4da54646aa9079d820192f21b1)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:42.581173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:42.591237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:42.592608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1007/2010] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:42.593467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d200e11-b37a-4e76-9dc5-1d2dd901301f service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:42.606584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:42.617448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d200e11-b37a-4e76-9dc5-1d2dd901301f service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:42.648126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8f6c2-55af-483e-bd2d-6261c3fb724f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:42.649907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8f6c2-55af-483e-bd2d-6261c3fb724f admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:42.650429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8f6c2-55af-483e-bd2d-6261c3fb724f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:42.650721 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8f6c2-55af-483e-bd2d-6261c3fb724f admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:42.652696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8f6c2-55af-483e-bd2d-6261c3fb724f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:42.661192 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8f6c2-55af-483e-bd2d-6261c3fb724f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:42.681826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1008/2011] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:42.684664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8f6c2-55af-483e-bd2d-6261c3fb724f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:42.693662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab884a4da54646aa9079d820192f21b1', 'request_id': 'req-5e6b136d-5620-4528-8b4e-31505eda7358'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:42.700215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8f6c2-55af-483e-bd2d-6261c3fb724f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:42.715824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab884a4da54646aa9079d820192f21b1', 'request_id': 'req-5e6b136d-5620-4528-8b4e-31505eda7358'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:42.733974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab884a4da54646aa9079d820192f21b1', 'request_id': 'req-5e6b136d-5620-4528-8b4e-31505eda7358'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:42.766405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] The token cache is being invalidate because project ab884a4da54646aa9079d820192f21b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:42.766793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:42.772755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:42.777070 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1004/2012] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:42 2022] DELETE /identity/v3/projects/ab884a4da54646aa9079d820192f21b1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:42.780841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:42.790176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:42.812041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753dd35-b239-4f1e-a6ca-e53ff999ce21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:42.812963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753dd35-b239-4f1e-a6ca-e53ff999ce21 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:42.813611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753dd35-b239-4f1e-a6ca-e53ff999ce21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:42.814118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753dd35-b239-4f1e-a6ca-e53ff999ce21 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:42.815156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753dd35-b239-4f1e-a6ca-e53ff999ce21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:42.818545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753dd35-b239-4f1e-a6ca-e53ff999ce21 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:42.830907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1009/2013] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:42.835801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753dd35-b239-4f1e-a6ca-e53ff999ce21 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:42.844298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c753dd35-b239-4f1e-a6ca-e53ff999ce21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:42.873572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753dd35-b239-4f1e-a6ca-e53ff999ce21 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:42.885897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d27020-a87a-405b-b6b0-c08d89ce39b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:42.886687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d27020-a87a-405b-b6b0-c08d89ce39b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:42.887070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d27020-a87a-405b-b6b0-c08d89ce39b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:42.887486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d27020-a87a-405b-b6b0-c08d89ce39b9 admin admin] PATH_INFO: `/v3/projects/410d017c5fd24ac98506e3f71bf851e9/users/83d3992eb69540a5b87c3857376c82ef/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:42.907258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d27020-a87a-405b-b6b0-c08d89ce39b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=410d017c5fd24ac98506e3f71bf851e9, user_id=83d3992eb69540a5b87c3857376c82ef, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:42.912086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d27020-a87a-405b-b6b0-c08d89ce39b9 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:42.923750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b136d-5620-4528-8b4e-31505eda7358 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:42.945533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1010/2014] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:42 2022] PUT /identity/v3/projects/410d017c5fd24ac98506e3f71bf851e9/users/83d3992eb69540a5b87c3857376c82ef/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:42.948400 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12438b52-06e9-43d2-b93e-ee441234ed00 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:42.948845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12438b52-06e9-43d2-b93e-ee441234ed00 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:42.949235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12438b52-06e9-43d2-b93e-ee441234ed00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:43.040308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-12438b52-06e9-43d2-b93e-ee441234ed00 None None] MFA Rules not processed for user `83d3992eb69540a5b87c3857376c82ef`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:43.047512 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-12438b52-06e9-43d2-b93e-ee441234ed00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:43.078210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:43.079112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:43.079687 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:43.080169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] PATH_INFO: `/v3/users/cc05ba3ebaaa471b80a4a0d14c91fcf4` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:43.083396 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc05ba3ebaaa471b80a4a0d14c91fcf4)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:43.085830 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:43.181722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:43.189891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc05ba3ebaaa471b80a4a0d14c91fcf4'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:43.208413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc05ba3ebaaa471b80a4a0d14c91fcf4'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:43.209557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc05ba3ebaaa471b80a4a0d14c91fcf4'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:43.217454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1011/2015] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:43.224265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12438b52-06e9-43d2-b93e-ee441234ed00 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:43.245026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12438b52-06e9-43d2-b93e-ee441234ed00 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:43.247742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc05ba3ebaaa471b80a4a0d14c91fcf4'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:43.248593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc05ba3ebaaa471b80a4a0d14c91fcf4'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:43.263374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a2c23-eb54-415b-bcd1-d35ddd7e484c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:43.264257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a2c23-eb54-415b-bcd1-d35ddd7e484c admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:43.264706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a2c23-eb54-415b-bcd1-d35ddd7e484c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:43.265248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a2c23-eb54-415b-bcd1-d35ddd7e484c admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:43.266173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a2c23-eb54-415b-bcd1-d35ddd7e484c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:43.270805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a2c23-eb54-415b-bcd1-d35ddd7e484c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:43.273057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1005/2016] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:42 2022] DELETE /identity/v3/users/cc05ba3ebaaa471b80a4a0d14c91fcf4 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:43.353565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1012/2017] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:43 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:43.358631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:43.360470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:43.556477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417339e-2f23-4ccf-aabc-9ea10ac32561 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:43.604500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a2c23-eb54-415b-bcd1-d35ddd7e484c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:43.607930 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4a0a2c23-eb54-415b-bcd1-d35ddd7e484c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:43.686246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3640c6-007d-4db9-8ed3-4113ddf7f5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:43.687100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3640c6-007d-4db9-8ed3-4113ddf7f5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:43.688051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3640c6-007d-4db9-8ed3-4113ddf7f5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:43.688376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3640c6-007d-4db9-8ed3-4113ddf7f5b1 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:43.691598 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3640c6-007d-4db9-8ed3-4113ddf7f5b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:43.695850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3640c6-007d-4db9-8ed3-4113ddf7f5b1 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:43.721460 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:43.758379 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1006/2018] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:43 2022] POST /identity/v3/users => generated 300 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:43.763849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3640c6-007d-4db9-8ed3-4113ddf7f5b1 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:43.791645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3640c6-007d-4db9-8ed3-4113ddf7f5b1 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:43.794280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a2c23-eb54-415b-bcd1-d35ddd7e484c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:43.800924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:43.801359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2900e19-5686-424a-9f88-21598bf88454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:43.801601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:43.801880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:43.802175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] PATH_INFO: `/v3/users/592a5d46fd81440d92f9167bf6002831` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:43.803160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2900e19-5686-424a-9f88-21598bf88454 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:43.804218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2900e19-5686-424a-9f88-21598bf88454 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:43.804464 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2900e19-5686-424a-9f88-21598bf88454 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:43.805298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=592a5d46fd81440d92f9167bf6002831)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:43.806972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2900e19-5686-424a-9f88-21598bf88454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:43.808546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:43.817981 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2900e19-5686-424a-9f88-21598bf88454 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:43.844004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1007/2019] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:43.849290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2900e19-5686-424a-9f88-21598bf88454 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:43.861174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2900e19-5686-424a-9f88-21598bf88454 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:43.868159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff37e11-c938-4de4-94a3-40b7f3d58c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:43.868159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff37e11-c938-4de4-94a3-40b7f3d58c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:43.868506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff37e11-c938-4de4-94a3-40b7f3d58c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:43.868629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff37e11-c938-4de4-94a3-40b7f3d58c1e admin admin] PATH_INFO: `/v3/projects/6e8b3a1f5f92400cb190f7c8b1158961/users/387b8ec5932d488cbb5ed40d56e6ff31/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:43.889167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff37e11-c938-4de4-94a3-40b7f3d58c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8b3a1f5f92400cb190f7c8b1158961, user_id=387b8ec5932d488cbb5ed40d56e6ff31, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:43.892503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff37e11-c938-4de4-94a3-40b7f3d58c1e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:43.894389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592a5d46fd81440d92f9167bf6002831'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:43.922997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592a5d46fd81440d92f9167bf6002831'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:43.924789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592a5d46fd81440d92f9167bf6002831'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:43.925853 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592a5d46fd81440d92f9167bf6002831'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:43.926061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1008/2020] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:43 2022] PUT /identity/v3/projects/6e8b3a1f5f92400cb190f7c8b1158961/users/387b8ec5932d488cbb5ed40d56e6ff31/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:43.927834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff37e11-c938-4de4-94a3-40b7f3d58c1e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:43.929593 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-dff37e11-c938-4de4-94a3-40b7f3d58c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:43.936425 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592a5d46fd81440d92f9167bf6002831'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:43.964637 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1013/2021] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:43 2022] DELETE /identity/v3/users/592a5d46fd81440d92f9167bf6002831 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:43.964637 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:43.964637 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:43.996536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:44.128839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff37e11-c938-4de4-94a3-40b7f3d58c1e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:44.138537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:44.139486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:44.140244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:44.140723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] PATH_INFO: `/v3/projects/6fe35eec17a94e8ba8670d5f08f36ec4` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:44.142609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fe35eec17a94e8ba8670d5f08f36ec4)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:44.147850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69064e9-0a52-4307-ba16-b7228b0c8828 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:44.154192 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc3c2c-0032-4a88-bf13-9122a432bed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:44.154898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc3c2c-0032-4a88-bf13-9122a432bed3 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:44.155307 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc3c2c-0032-4a88-bf13-9122a432bed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:44.155577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc3c2c-0032-4a88-bf13-9122a432bed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:44.156243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc3c2c-0032-4a88-bf13-9122a432bed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:44.158233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc3c2c-0032-4a88-bf13-9122a432bed3 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:44.164430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:44.174046 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1014/2022] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:44.175417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee6ebc-2a24-48da-a680-dca5e66c7b52 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:44.175707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee6ebc-2a24-48da-a680-dca5e66c7b52 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:44.175980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee6ebc-2a24-48da-a680-dca5e66c7b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:44.197747 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-d1ee6ebc-2a24-48da-a680-dca5e66c7b52 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:44.199314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d1ee6ebc-2a24-48da-a680-dca5e66c7b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:44.212320 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1015/2023] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:44.213362 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee6ebc-2a24-48da-a680-dca5e66c7b52 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:44.226488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee6ebc-2a24-48da-a680-dca5e66c7b52 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:44.241377 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fe35eec17a94e8ba8670d5f08f36ec4', 'request_id': 'req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:44.250568 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fe35eec17a94e8ba8670d5f08f36ec4', 'request_id': 'req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:44.260142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fe35eec17a94e8ba8670d5f08f36ec4', 'request_id': 'req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:44.277625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] The token cache is being invalidate because project 6fe35eec17a94e8ba8670d5f08f36ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:44.277986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:44.278897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:44.280957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1009/2024] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:43 2022] DELETE /identity/v3/projects/6fe35eec17a94e8ba8670d5f08f36ec4 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:44.282111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:44.283809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:44.317779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:44.350976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52c9df-e624-42e5-9b87-2a442e2c8d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:44.351783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52c9df-e624-42e5-9b87-2a442e2c8d71 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:44.352069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52c9df-e624-42e5-9b87-2a442e2c8d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:44.352404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52c9df-e624-42e5-9b87-2a442e2c8d71 admin admin] PATH_INFO: `/v3/projects/6e8b3a1f5f92400cb190f7c8b1158961/users/387b8ec5932d488cbb5ed40d56e6ff31/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:44.354862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52c9df-e624-42e5-9b87-2a442e2c8d71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8b3a1f5f92400cb190f7c8b1158961, user_id=387b8ec5932d488cbb5ed40d56e6ff31, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:44.358082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52c9df-e624-42e5-9b87-2a442e2c8d71 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:44.374772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1016/2025] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:44 2022] PUT /identity/v3/projects/6e8b3a1f5f92400cb190f7c8b1158961/users/387b8ec5932d488cbb5ed40d56e6ff31/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:44.386312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829271e2-5787-4be5-b2ad-e7294aa1dfac None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:44.386605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829271e2-5787-4be5-b2ad-e7294aa1dfac None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:44.386936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829271e2-5787-4be5-b2ad-e7294aa1dfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:44.422150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5d0e2-f650-46dd-9ae0-b4ebddd4e764 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:44.425527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-829271e2-5787-4be5-b2ad-e7294aa1dfac None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:44.428381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-829271e2-5787-4be5-b2ad-e7294aa1dfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:44.541999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1017/2026] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:44.544716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b17fca-0933-4577-8f28-55f66190fe07 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:44.545204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b17fca-0933-4577-8f28-55f66190fe07 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:44.545894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b17fca-0933-4577-8f28-55f66190fe07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:44.555629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794b0a5-96d6-4be2-a2f8-df897aec92d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:44.556322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794b0a5-96d6-4be2-a2f8-df897aec92d0 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:44.556597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794b0a5-96d6-4be2-a2f8-df897aec92d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:44.556864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794b0a5-96d6-4be2-a2f8-df897aec92d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:44.558769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2794b0a5-96d6-4be2-a2f8-df897aec92d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:44.574741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794b0a5-96d6-4be2-a2f8-df897aec92d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:44.576339 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794b0a5-96d6-4be2-a2f8-df897aec92d0 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:44.587777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1010/2027] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:44.590014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794b0a5-96d6-4be2-a2f8-df897aec92d0 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:44.590014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2794b0a5-96d6-4be2-a2f8-df897aec92d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:44.604023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-20b17fca-0933-4577-8f28-55f66190fe07 None None] MFA Rules not processed for user `387b8ec5932d488cbb5ed40d56e6ff31`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:44.607959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-20b17fca-0933-4577-8f28-55f66190fe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:44.613137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794b0a5-96d6-4be2-a2f8-df897aec92d0 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:44.626881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:44.629481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:44.629696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:44.629996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] PATH_INFO: `/v3/projects/03de73fafdda4985ae0bf5b7173a1482` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:44.633824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03de73fafdda4985ae0bf5b7173a1482)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:44.636294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:44.673457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:44.747648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1018/2028] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:44.754744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e2dc4-aabd-4557-8787-8fa7d5750a67 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:44.754943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e2dc4-aabd-4557-8787-8fa7d5750a67 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:44.755090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e2dc4-aabd-4557-8787-8fa7d5750a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:44.780701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-135e2dc4-aabd-4557-8787-8fa7d5750a67 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:44.782301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-135e2dc4-aabd-4557-8787-8fa7d5750a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:44.806654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03de73fafdda4985ae0bf5b7173a1482', 'request_id': 'req-1c14198a-538c-4efb-81ab-5c34f4ee4c83'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:44.826974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03de73fafdda4985ae0bf5b7173a1482', 'request_id': 'req-1c14198a-538c-4efb-81ab-5c34f4ee4c83'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:44.834250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03de73fafdda4985ae0bf5b7173a1482', 'request_id': 'req-1c14198a-538c-4efb-81ab-5c34f4ee4c83'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:44.853061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] The token cache is being invalidate because project 03de73fafdda4985ae0bf5b7173a1482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:44.853439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:44.854559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:44.858953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1011/2029] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:44 2022] DELETE /identity/v3/projects/03de73fafdda4985ae0bf5b7173a1482 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:44.858953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:44.858953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:44.950969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14198a-538c-4efb-81ab-5c34f4ee4c83 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:44.962572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3efce9-a3fb-43f4-9c28-1d17afdec887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:44.963493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3efce9-a3fb-43f4-9c28-1d17afdec887 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:44.963876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3efce9-a3fb-43f4-9c28-1d17afdec887 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:44.964233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3efce9-a3fb-43f4-9c28-1d17afdec887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:44.968065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ba3efce9-a3fb-43f4-9c28-1d17afdec887 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:44.997477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:45.026940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1019/2030] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:45.028877 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e2dc4-aabd-4557-8787-8fa7d5750a67 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:45.031420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-135e2dc4-aabd-4557-8787-8fa7d5750a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:45.059916 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e2dc4-aabd-4557-8787-8fa7d5750a67 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:45.069973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:45.071300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:45.071836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:45.072326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] PATH_INFO: `/v3/users/14053590184f4e7dbbef122b3822fefe` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:45.075533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=14053590184f4e7dbbef122b3822fefe)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:45.079034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:45.120308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3efce9-a3fb-43f4-9c28-1d17afdec887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:45.124978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3efce9-a3fb-43f4-9c28-1d17afdec887 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:45.147027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1012/2031] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:45.148079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3efce9-a3fb-43f4-9c28-1d17afdec887 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:45.162168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3efce9-a3fb-43f4-9c28-1d17afdec887 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:45.171941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587300e-7d10-4d6e-9df8-1993b7007e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:45.171941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587300e-7d10-4d6e-9df8-1993b7007e0c admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:45.171941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587300e-7d10-4d6e-9df8-1993b7007e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:45.171941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587300e-7d10-4d6e-9df8-1993b7007e0c admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:45.173028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587300e-7d10-4d6e-9df8-1993b7007e0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:45.173028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587300e-7d10-4d6e-9df8-1993b7007e0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:45.174543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587300e-7d10-4d6e-9df8-1993b7007e0c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:45.194764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14053590184f4e7dbbef122b3822fefe'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.205031 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1013/2032] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:45.211831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587300e-7d10-4d6e-9df8-1993b7007e0c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:45.229281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14053590184f4e7dbbef122b3822fefe'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.230076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14053590184f4e7dbbef122b3822fefe'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.230710 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14053590184f4e7dbbef122b3822fefe'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.242806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587300e-7d10-4d6e-9df8-1993b7007e0c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:45.244656 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14053590184f4e7dbbef122b3822fefe'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.249251 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75e4fb-773f-4708-9724-2232fca70d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:45.250129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75e4fb-773f-4708-9724-2232fca70d8e admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:45.250454 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75e4fb-773f-4708-9724-2232fca70d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:45.250625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75e4fb-773f-4708-9724-2232fca70d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:45.251371 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75e4fb-773f-4708-9724-2232fca70d8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:45.254411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75e4fb-773f-4708-9724-2232fca70d8e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:45.277489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1020/2033] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:45 2022] DELETE /identity/v3/users/14053590184f4e7dbbef122b3822fefe => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:45.285252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:45.287802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:45.307812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1014/2034] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:45 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:45.312000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75e4fb-773f-4708-9724-2232fca70d8e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:45.317960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7e75e4fb-773f-4708-9724-2232fca70d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:45.350027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:45.410965 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054d88d-b847-498a-81dc-263d1c3b4b8d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:45.530664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75e4fb-773f-4708-9724-2232fca70d8e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:45.547350 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076a6a9-a7d5-4d83-b2c3-eea5be185a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:45.548035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076a6a9-a7d5-4d83-b2c3-eea5be185a4c admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:45.548425 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076a6a9-a7d5-4d83-b2c3-eea5be185a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:45.548708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076a6a9-a7d5-4d83-b2c3-eea5be185a4c admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:45.549364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076a6a9-a7d5-4d83-b2c3-eea5be185a4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:45.552102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076a6a9-a7d5-4d83-b2c3-eea5be185a4c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:45.593376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1015/2035] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:45 2022] POST /identity/v3/users => generated 293 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:45.593376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076a6a9-a7d5-4d83-b2c3-eea5be185a4c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:45.600101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1076a6a9-a7d5-4d83-b2c3-eea5be185a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:45.623108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:45.624125 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:45.625235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:45.625235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] PATH_INFO: `/v3/users/83d3992eb69540a5b87c3857376c82ef` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:45.626207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d3992eb69540a5b87c3857376c82ef)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:45.629274 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:45.637922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076a6a9-a7d5-4d83-b2c3-eea5be185a4c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:45.653193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:45.654020 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:45.654304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:45.654579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] PATH_INFO: `/v3/projects/79f580f598c8478a964c189dd441a841` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:45.657576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f580f598c8478a964c189dd441a841)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:45.661422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:45.665631 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:45.710816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d3992eb69540a5b87c3857376c82ef'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.759285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d3992eb69540a5b87c3857376c82ef'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.760047 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d3992eb69540a5b87c3857376c82ef'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.760917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d3992eb69540a5b87c3857376c82ef'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.778194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d3992eb69540a5b87c3857376c82ef'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.785143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f580f598c8478a964c189dd441a841', 'request_id': 'req-f2388e03-1f10-4cb5-94de-150e96c5749f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.805576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f580f598c8478a964c189dd441a841', 'request_id': 'req-f2388e03-1f10-4cb5-94de-150e96c5749f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.817768 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1021/2036] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:45 2022] DELETE /identity/v3/users/83d3992eb69540a5b87c3857376c82ef => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:45.820291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:45.822245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:45.830211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f580f598c8478a964c189dd441a841', 'request_id': 'req-f2388e03-1f10-4cb5-94de-150e96c5749f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.862215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] The token cache is being invalidate because project 79f580f598c8478a964c189dd441a841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:45.862667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.865630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:45.870506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1016/2037] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:45 2022] DELETE /identity/v3/projects/79f580f598c8478a964c189dd441a841 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:45.874744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:45.876726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:46.023520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2f0e2-a11c-4544-b245-f545bab07a7a admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:46.035339 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5eb82-c6f0-4dfd-9f58-05bd5087c5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:46.035339 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5eb82-c6f0-4dfd-9f58-05bd5087c5dd admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:46.036273 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5eb82-c6f0-4dfd-9f58-05bd5087c5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:46.036628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5eb82-c6f0-4dfd-9f58-05bd5087c5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:46.037591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5eb82-c6f0-4dfd-9f58-05bd5087c5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:46.039698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5eb82-c6f0-4dfd-9f58-05bd5087c5dd admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:46.044617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2388e03-1f10-4cb5-94de-150e96c5749f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:46.061000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:46.061928 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:46.062398 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:46.062935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] PATH_INFO: `/v3/projects/216fac093d094f708e659f5927204abb` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:46.066346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=216fac093d094f708e659f5927204abb)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:46.070515 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:46.090582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1022/2038] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:45 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:46.091181 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df9ccd-e700-44a1-9421-0097599e0daf None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:46.091842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df9ccd-e700-44a1-9421-0097599e0daf None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:46.092777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df9ccd-e700-44a1-9421-0097599e0daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:46.121798 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:46.139105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-86df9ccd-e700-44a1-9421-0097599e0daf None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:46.140958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-86df9ccd-e700-44a1-9421-0097599e0daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:46.177546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '216fac093d094f708e659f5927204abb', 'request_id': 'req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:46.199477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '216fac093d094f708e659f5927204abb', 'request_id': 'req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:46.213672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '216fac093d094f708e659f5927204abb', 'request_id': 'req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:46.235165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] The token cache is being invalidate because project 216fac093d094f708e659f5927204abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:46.235482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:46.236194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:46.238533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1017/2039] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:45 2022] DELETE /identity/v3/projects/216fac093d094f708e659f5927204abb => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:46.239086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:46.240541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:46.346695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1023/2040] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:46.349950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df9ccd-e700-44a1-9421-0097599e0daf None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:46.361580 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-86df9ccd-e700-44a1-9421-0097599e0daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:46.414088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed6cbd-6ea3-42e1-8d67-f1bd6691eef8 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:46.416856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df9ccd-e700-44a1-9421-0097599e0daf None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:46.424584 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:46.426702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:46.426986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:46.427299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] PATH_INFO: `/v3/users/2ac12a69255a4c4fa00f9ad35e248950` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:46.430198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac12a69255a4c4fa00f9ad35e248950)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:46.435855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:46.437121 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126b032-6852-4921-8bac-92d22575e269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:46.439663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126b032-6852-4921-8bac-92d22575e269 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:46.440352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126b032-6852-4921-8bac-92d22575e269 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:46.440530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126b032-6852-4921-8bac-92d22575e269 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:46.442813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126b032-6852-4921-8bac-92d22575e269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:46.452040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126b032-6852-4921-8bac-92d22575e269 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:46.500870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1024/2041] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:46.507915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87668e8e-3108-4cff-be94-dba6ff75118a None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:46.507915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87668e8e-3108-4cff-be94-dba6ff75118a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:46.507915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87668e8e-3108-4cff-be94-dba6ff75118a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:46.551671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac12a69255a4c4fa00f9ad35e248950'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:46.567742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-87668e8e-3108-4cff-be94-dba6ff75118a None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:46.568952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-87668e8e-3108-4cff-be94-dba6ff75118a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:46.574188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac12a69255a4c4fa00f9ad35e248950'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:46.578697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac12a69255a4c4fa00f9ad35e248950'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:46.593011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1025/2042] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:46.593985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87668e8e-3108-4cff-be94-dba6ff75118a None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:46.595667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-87668e8e-3108-4cff-be94-dba6ff75118a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:46.619930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac12a69255a4c4fa00f9ad35e248950'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:46.620794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac12a69255a4c4fa00f9ad35e248950'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:46.636895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:46.651018 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1018/2043] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:46 2022] DELETE /identity/v3/users/2ac12a69255a4c4fa00f9ad35e248950 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:46.652386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:46.663884 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:46.698951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87668e8e-3108-4cff-be94-dba6ff75118a None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:46.754389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:46.779244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8470d-fb2a-43bc-bf24-304758105a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:46.779929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8470d-fb2a-43bc-bf24-304758105a79 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:46.780801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8470d-fb2a-43bc-bf24-304758105a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:46.781189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8470d-fb2a-43bc-bf24-304758105a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:46.785625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-f0a8470d-fb2a-43bc-bf24-304758105a79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:46.835191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e181b-bc74-4d7f-979b-50eaea49dc71 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:46.856908 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bb4e0-eeba-4246-9449-0349f3cb9a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:46.858993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bb4e0-eeba-4246-9449-0349f3cb9a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:46.860234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bb4e0-eeba-4246-9449-0349f3cb9a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:46.862075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bb4e0-eeba-4246-9449-0349f3cb9a7c admin admin] PATH_INFO: `/v3/projects/97c88f12aeea4bed8ed0b3e3c58b7205/users/290debe72e0a4630ae3e1481a8f24453/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:46.883156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8470d-fb2a-43bc-bf24-304758105a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:46.885385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8470d-fb2a-43bc-bf24-304758105a79 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:46.900791 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1026/2044] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:46.902340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cd3d8-74c3-4fcf-931c-23fb40f1f3da None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:46.902891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cd3d8-74c3-4fcf-931c-23fb40f1f3da None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:46.903304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cd3d8-74c3-4fcf-931c-23fb40f1f3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:46.914518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016bb4e0-eeba-4246-9449-0349f3cb9a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c88f12aeea4bed8ed0b3e3c58b7205, user_id=290debe72e0a4630ae3e1481a8f24453, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:46.936411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016bb4e0-eeba-4246-9449-0349f3cb9a7c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:46.942060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-3f3cd3d8-74c3-4fcf-931c-23fb40f1f3da None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:46.943697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3f3cd3d8-74c3-4fcf-931c-23fb40f1f3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:46.953705 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1027/2045] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:46.958543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cd3d8-74c3-4fcf-931c-23fb40f1f3da None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:46.981592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1019/2046] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:46 2022] PUT /identity/v3/projects/97c88f12aeea4bed8ed0b3e3c58b7205/users/290debe72e0a4630ae3e1481a8f24453/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:46.989044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c326-2943-4a61-8ea6-507d8146303d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:46.989850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c326-2943-4a61-8ea6-507d8146303d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:46.990292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c326-2943-4a61-8ea6-507d8146303d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:46.991427 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cd3d8-74c3-4fcf-931c-23fb40f1f3da None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:47.078207 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-7b95c326-2943-4a61-8ea6-507d8146303d None None] MFA Rules not processed for user `290debe72e0a4630ae3e1481a8f24453`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:47.080676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7b95c326-2943-4a61-8ea6-507d8146303d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:47.125685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010ebed-5629-40d5-ae47-2bb99919f9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:47.127353 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010ebed-5629-40d5-ae47-2bb99919f9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:47.128847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010ebed-5629-40d5-ae47-2bb99919f9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:47.129707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010ebed-5629-40d5-ae47-2bb99919f9d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:47.131554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010ebed-5629-40d5-ae47-2bb99919f9d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:47.133196 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010ebed-5629-40d5-ae47-2bb99919f9d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:47.139623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010ebed-5629-40d5-ae47-2bb99919f9d2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:47.173886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1028/2047] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:47.179010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010ebed-5629-40d5-ae47-2bb99919f9d2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:47.194581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010ebed-5629-40d5-ae47-2bb99919f9d2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:47.199714 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc37191-f74f-4de4-a5e5-031a936f0687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:47.200333 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc37191-f74f-4de4-a5e5-031a936f0687 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:47.200860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc37191-f74f-4de4-a5e5-031a936f0687 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:47.201160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc37191-f74f-4de4-a5e5-031a936f0687 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:47.201813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc37191-f74f-4de4-a5e5-031a936f0687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:47.203914 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc37191-f74f-4de4-a5e5-031a936f0687 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:47.216149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:47.243669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1029/2048] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:47 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:47.243920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1020/2049] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:47.244072 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc37191-f74f-4de4-a5e5-031a936f0687 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:47.245646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95c326-2943-4a61-8ea6-507d8146303d None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:47.264470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95c326-2943-4a61-8ea6-507d8146303d None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:47.351781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc37191-f74f-4de4-a5e5-031a936f0687 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:47.357050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2174db-f2fb-4906-a4c3-4b84bbea7ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:47.358034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2174db-f2fb-4906-a4c3-4b84bbea7ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:47.358694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2174db-f2fb-4906-a4c3-4b84bbea7ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:47.359584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2174db-f2fb-4906-a4c3-4b84bbea7ae3 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:47.359905 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2174db-f2fb-4906-a4c3-4b84bbea7ae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:47.362046 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2174db-f2fb-4906-a4c3-4b84bbea7ae3 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:47.396675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1030/2050] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:47 2022] POST /identity/v3/users => generated 285 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:47.400840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2174db-f2fb-4906-a4c3-4b84bbea7ae3 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:47.402887 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7b2174db-f2fb-4906-a4c3-4b84bbea7ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:47.403822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25d7c5-54f1-4023-85a8-d95a400565a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:47.405057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25d7c5-54f1-4023-85a8-d95a400565a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:47.405576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25d7c5-54f1-4023-85a8-d95a400565a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:47.406123 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25d7c5-54f1-4023-85a8-d95a400565a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:47.407923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25d7c5-54f1-4023-85a8-d95a400565a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:47.414837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25d7c5-54f1-4023-85a8-d95a400565a4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:47.434921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2174db-f2fb-4906-a4c3-4b84bbea7ae3 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:47.447718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:47.449138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:47.451993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:47.452747 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] PATH_INFO: `/v3/projects/1452591191f94e55b9c4a313d9dee8c0` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:47.456561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1452591191f94e55b9c4a313d9dee8c0)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:47.459841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:47.471749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1021/2051] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:47 2022] POST /identity/v3/projects => generated 383 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:47.471749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25d7c5-54f1-4023-85a8-d95a400565a4 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:47.514749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:47.564323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25d7c5-54f1-4023-85a8-d95a400565a4 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:47.578426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1452591191f94e55b9c4a313d9dee8c0', 'request_id': 'req-b09b8d35-9096-48b8-9668-f58ee6d11ab0'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:47.592154 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1452591191f94e55b9c4a313d9dee8c0', 'request_id': 'req-b09b8d35-9096-48b8-9668-f58ee6d11ab0'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:47.604907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1452591191f94e55b9c4a313d9dee8c0', 'request_id': 'req-b09b8d35-9096-48b8-9668-f58ee6d11ab0'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:47.630289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] The token cache is being invalidate because project 1452591191f94e55b9c4a313d9dee8c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:47.631586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:47.632118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:47.640392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1031/2052] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:47 2022] DELETE /identity/v3/projects/1452591191f94e55b9c4a313d9dee8c0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:47.642399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:47.645381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:47.708091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193bb5c-421c-488d-80fd-244b1da8ac0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:47.708733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193bb5c-421c-488d-80fd-244b1da8ac0f admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:47.709011 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193bb5c-421c-488d-80fd-244b1da8ac0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:47.709281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193bb5c-421c-488d-80fd-244b1da8ac0f admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:47.709943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193bb5c-421c-488d-80fd-244b1da8ac0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:47.712366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193bb5c-421c-488d-80fd-244b1da8ac0f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:47.725157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1022/2053] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:47.726520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a037ac7-d63a-439d-9abb-9dac9bad8e53 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:47.726806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a037ac7-d63a-439d-9abb-9dac9bad8e53 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:47.729500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a037ac7-d63a-439d-9abb-9dac9bad8e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:47.766603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-6a037ac7-d63a-439d-9abb-9dac9bad8e53 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:47.771037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6a037ac7-d63a-439d-9abb-9dac9bad8e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:47.804765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1023/2054] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:47.812681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a037ac7-d63a-439d-9abb-9dac9bad8e53 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:47.815469 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6a037ac7-d63a-439d-9abb-9dac9bad8e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:47.822101 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b8d35-9096-48b8-9668-f58ee6d11ab0 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:47.829287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abdbc64-0c71-4a7c-81c6-2c812369a191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:47.830098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdbc64-0c71-4a7c-81c6-2c812369a191 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:47.830331 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdbc64-0c71-4a7c-81c6-2c812369a191 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:47.830609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdbc64-0c71-4a7c-81c6-2c812369a191 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:47.831668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abdbc64-0c71-4a7c-81c6-2c812369a191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:47.833844 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abdbc64-0c71-4a7c-81c6-2c812369a191 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:47.835503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a037ac7-d63a-439d-9abb-9dac9bad8e53 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:47.841590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62792fc4-bfb8-4c5c-a56b-1dbf62edbd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:47.848665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62792fc4-bfb8-4c5c-a56b-1dbf62edbd95 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:47.848665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62792fc4-bfb8-4c5c-a56b-1dbf62edbd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:47.848665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62792fc4-bfb8-4c5c-a56b-1dbf62edbd95 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:47.848665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62792fc4-bfb8-4c5c-a56b-1dbf62edbd95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:47.848665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62792fc4-bfb8-4c5c-a56b-1dbf62edbd95 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:47.853736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:47.869129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1024/2055] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:47.870365 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62792fc4-bfb8-4c5c-a56b-1dbf62edbd95 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:47.872278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-62792fc4-bfb8-4c5c-a56b-1dbf62edbd95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:47.880074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1032/2056] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:47 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:47.880074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72dfca-6586-46ff-a9d7-6421d08e228c None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:47.880074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72dfca-6586-46ff-a9d7-6421d08e228c None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:47.880074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72dfca-6586-46ff-a9d7-6421d08e228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:47.891376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62792fc4-bfb8-4c5c-a56b-1dbf62edbd95 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:47.897782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:47.898591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:47.899340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:47.899340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] PATH_INFO: `/v3/projects/410d017c5fd24ac98506e3f71bf851e9` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:47.901042 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] RBAC: Authorizing `identity:delete_project(project_id=410d017c5fd24ac98506e3f71bf851e9)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:47.903423 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-8a72dfca-6586-46ff-a9d7-6421d08e228c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:47.903990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:47.904724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-8a72dfca-6586-46ff-a9d7-6421d08e228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:47.921067 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1033/2057] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:47.922643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72dfca-6586-46ff-a9d7-6421d08e228c None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:47.938166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72dfca-6586-46ff-a9d7-6421d08e228c None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:47.944233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81525412-21a5-46f3-a62c-ee8ec6c223ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:47.944906 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81525412-21a5-46f3-a62c-ee8ec6c223ef admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:47.945180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81525412-21a5-46f3-a62c-ee8ec6c223ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:47.945451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81525412-21a5-46f3-a62c-ee8ec6c223ef admin admin] PATH_INFO: `/v3/projects/1cdc3c67b3ae461ab319c5e3530dc4bc/users/8d19f42081f3422c947cfbabc40d94ea/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:47.966554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81525412-21a5-46f3-a62c-ee8ec6c223ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cdc3c67b3ae461ab319c5e3530dc4bc, user_id=8d19f42081f3422c947cfbabc40d94ea, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:47.970134 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81525412-21a5-46f3-a62c-ee8ec6c223ef admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:47.983543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '410d017c5fd24ac98506e3f71bf851e9', 'request_id': 'req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:47.986642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1034/2058] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:47 2022] PUT /identity/v3/projects/1cdc3c67b3ae461ab319c5e3530dc4bc/users/8d19f42081f3422c947cfbabc40d94ea/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:47.987843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81525412-21a5-46f3-a62c-ee8ec6c223ef admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:47.990611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '410d017c5fd24ac98506e3f71bf851e9', 'request_id': 'req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:47.996856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '410d017c5fd24ac98506e3f71bf851e9', 'request_id': 'req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:48.014595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] The token cache is being invalidate because project 410d017c5fd24ac98506e3f71bf851e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:48.015426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:48.016213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:48.017922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1025/2059] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:47 2022] DELETE /identity/v3/projects/410d017c5fd24ac98506e3f71bf851e9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:48.019656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:48.021231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:48.070165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:48.084773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81525412-21a5-46f3-a62c-ee8ec6c223ef admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:48.092188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942e450-2747-44fb-a5a1-600eee81ccb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:48.092813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942e450-2747-44fb-a5a1-600eee81ccb1 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.093097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942e450-2747-44fb-a5a1-600eee81ccb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.093369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942e450-2747-44fb-a5a1-600eee81ccb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:48.094017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942e450-2747-44fb-a5a1-600eee81ccb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:48.095732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942e450-2747-44fb-a5a1-600eee81ccb1 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:48.099691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b1708-ed3b-4cdf-9a14-1fcb15c67756 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:48.111231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1035/2060] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:48.111231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17b3e6-bf5e-484c-b968-4a5d9c37cfd6 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.111231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17b3e6-bf5e-484c-b968-4a5d9c37cfd6 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.111231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17b3e6-bf5e-484c-b968-4a5d9c37cfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:48.120881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f5c8d-5e22-4a4a-bfa4-ca339bde91f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:48.121524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f5c8d-5e22-4a4a-bfa4-ca339bde91f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.121807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f5c8d-5e22-4a4a-bfa4-ca339bde91f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.122107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f5c8d-5e22-4a4a-bfa4-ca339bde91f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:48.122802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f5c8d-5e22-4a4a-bfa4-ca339bde91f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:48.123898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f5c8d-5e22-4a4a-bfa4-ca339bde91f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:48.124770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f5c8d-5e22-4a4a-bfa4-ca339bde91f8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:48.144718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1026/2061] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:48.144718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f5c8d-5e22-4a4a-bfa4-ca339bde91f8 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:48.145072 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ef17b3e6-bf5e-484c-b968-4a5d9c37cfd6 None None] MFA Rules not processed for user `8d19f42081f3422c947cfbabc40d94ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:48.145239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ca8f5c8d-5e22-4a4a-bfa4-ca339bde91f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:48.145377 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ef17b3e6-bf5e-484c-b968-4a5d9c37cfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:48.158227 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f5c8d-5e22-4a4a-bfa4-ca339bde91f8 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:48.166529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:48.167216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.167499 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.167767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] PATH_INFO: `/v3/users/387b8ec5932d488cbb5ed40d56e6ff31` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:48.168954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=387b8ec5932d488cbb5ed40d56e6ff31)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:48.171041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:48.227885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1036/2062] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:48.228756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17b3e6-bf5e-484c-b968-4a5d9c37cfd6 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:48.229873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '387b8ec5932d488cbb5ed40d56e6ff31'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:48.230077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ef17b3e6-bf5e-484c-b968-4a5d9c37cfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:48.249017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '387b8ec5932d488cbb5ed40d56e6ff31'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:48.249741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '387b8ec5932d488cbb5ed40d56e6ff31'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:48.267453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17b3e6-bf5e-484c-b968-4a5d9c37cfd6 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:48.274936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e02cf-5b3f-447a-8109-e7e7605eb00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:48.275783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e02cf-5b3f-447a-8109-e7e7605eb00a admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.276479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e02cf-5b3f-447a-8109-e7e7605eb00a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.276953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e02cf-5b3f-447a-8109-e7e7605eb00a admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:48.278345 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e02cf-5b3f-447a-8109-e7e7605eb00a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:48.281541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '387b8ec5932d488cbb5ed40d56e6ff31'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:48.281705 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e02cf-5b3f-447a-8109-e7e7605eb00a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:48.282410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '387b8ec5932d488cbb5ed40d56e6ff31'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:48.297189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1027/2063] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:48 2022] DELETE /identity/v3/users/387b8ec5932d488cbb5ed40d56e6ff31 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:48.297889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:48.299312 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:48.303435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1037/2064] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:48.303435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e02cf-5b3f-447a-8109-e7e7605eb00a admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:48.306126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-dd6e02cf-5b3f-447a-8109-e7e7605eb00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:48.390099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:48.395809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fa13e-7ba8-4fc8-8ae0-a85960a42b2f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:48.403745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f1501-6a42-41ae-a687-a3b9868254f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:48.404372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1501-6a42-41ae-a687-a3b9868254f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.404661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1501-6a42-41ae-a687-a3b9868254f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.404938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1501-6a42-41ae-a687-a3b9868254f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:48.407750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f1501-6a42-41ae-a687-a3b9868254f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:48.412049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f1501-6a42-41ae-a687-a3b9868254f0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:48.455239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1028/2065] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:48 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:48.456850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f1501-6a42-41ae-a687-a3b9868254f0 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:48.459561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e02cf-5b3f-447a-8109-e7e7605eb00a admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:48.460330 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-810f1501-6a42-41ae-a687-a3b9868254f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:48.510945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:48.556724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd708080-2d97-44a8-8a10-f6767f6603d4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06fad9466df14827983dac2b4b844f3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:48.559481 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd708080-2d97-44a8-8a10-f6767f6603d4 service glance] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.563482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd708080-2d97-44a8-8a10-f6767f6603d4 service glance] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.564249 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd708080-2d97-44a8-8a10-f6767f6603d4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:48.569482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-cd708080-2d97-44a8-8a10-f6767f6603d4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:48.605336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f1501-6a42-41ae-a687-a3b9868254f0 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:48.611694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793f741-0589-4bb0-a54c-3166f6acf40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:48.613170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793f741-0589-4bb0-a54c-3166f6acf40b admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.615582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793f741-0589-4bb0-a54c-3166f6acf40b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.615989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793f741-0589-4bb0-a54c-3166f6acf40b admin admin] PATH_INFO: `/v3/projects/88a057a14c5d4411832317fb809dcd00/users/53ac7623b3b740179eb2022500ed2616/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:48.634276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793f741-0589-4bb0-a54c-3166f6acf40b admin admin] RBAC: Authorizing `identity:create_grant(project_id=88a057a14c5d4411832317fb809dcd00, user_id=53ac7623b3b740179eb2022500ed2616, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:48.637311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793f741-0589-4bb0-a54c-3166f6acf40b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:48.665431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1029/2066] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:48 2022] PUT /identity/v3/projects/88a057a14c5d4411832317fb809dcd00/users/53ac7623b3b740179eb2022500ed2616/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:48.665797 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd708080-2d97-44a8-8a10-f6767f6603d4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:48.665909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793f741-0589-4bb0-a54c-3166f6acf40b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:48.665997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd708080-2d97-44a8-8a10-f6767f6603d4 service glance] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:48.670025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1038/2067] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:48.671195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9a30c-98ba-4d99-9433-1cbf98d3c397 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.671447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9a30c-98ba-4d99-9433-1cbf98d3c397 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.671883 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9a30c-98ba-4d99-9433-1cbf98d3c397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:48.708437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-a1c9a30c-98ba-4d99-9433-1cbf98d3c397 None None] MFA Rules not processed for user `53ac7623b3b740179eb2022500ed2616`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:48.709850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a1c9a30c-98ba-4d99-9433-1cbf98d3c397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:48.796481 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1039/2068] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:48.802876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9a30c-98ba-4d99-9433-1cbf98d3c397 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:48.812738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793f741-0589-4bb0-a54c-3166f6acf40b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:48.815290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9a30c-98ba-4d99-9433-1cbf98d3c397 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:48.818769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afe1cc-3b8c-4e0b-8961-340c84a5bc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:48.820390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4c7d2-2189-4fa0-987a-1f7df836e61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:48.821158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4c7d2-2189-4fa0-987a-1f7df836e61c admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.821450 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4c7d2-2189-4fa0-987a-1f7df836e61c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.821726 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4c7d2-2189-4fa0-987a-1f7df836e61c admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:48.822722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afe1cc-3b8c-4e0b-8961-340c84a5bc17 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.822833 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4c7d2-2189-4fa0-987a-1f7df836e61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:48.823445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afe1cc-3b8c-4e0b-8961-340c84a5bc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.823792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afe1cc-3b8c-4e0b-8961-340c84a5bc17 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:48.824917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92afe1cc-3b8c-4e0b-8961-340c84a5bc17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:48.825221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4c7d2-2189-4fa0-987a-1f7df836e61c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:48.827112 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92afe1cc-3b8c-4e0b-8961-340c84a5bc17 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:48.843694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:48.867949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1040/2069] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:48 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:48.867949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4c7d2-2189-4fa0-987a-1f7df836e61c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:48.870015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3cd4c7d2-2189-4fa0-987a-1f7df836e61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:48.875383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1030/2070] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:48 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:48.875383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afe1cc-3b8c-4e0b-8961-340c84a5bc17 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:48.885132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afe1cc-3b8c-4e0b-8961-340c84a5bc17 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:48.966487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfd05a-fbc9-4f0c-9e6e-b7b68b7bd1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:48.967105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfd05a-fbc9-4f0c-9e6e-b7b68b7bd1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.967435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfd05a-fbc9-4f0c-9e6e-b7b68b7bd1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.967705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfd05a-fbc9-4f0c-9e6e-b7b68b7bd1f5 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:48.968356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addfd05a-fbc9-4f0c-9e6e-b7b68b7bd1f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:48.970619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addfd05a-fbc9-4f0c-9e6e-b7b68b7bd1f5 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:48.987584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4c7d2-2189-4fa0-987a-1f7df836e61c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:48.997518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:48.998253 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:48.998533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:48.998809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] PATH_INFO: `/v3/projects/6e8b3a1f5f92400cb190f7c8b1158961` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.000238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8b3a1f5f92400cb190f7c8b1158961)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:49.006949 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1031/2071] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:48 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:49.006949 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfd05a-fbc9-4f0c-9e6e-b7b68b7bd1f5 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:49.007272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:49.018560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfd05a-fbc9-4f0c-9e6e-b7b68b7bd1f5 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:49.025124 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a9ea1-ac1a-4d2f-abd9-86d98db1cfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:49.026273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a9ea1-ac1a-4d2f-abd9-86d98db1cfff admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.026612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a9ea1-ac1a-4d2f-abd9-86d98db1cfff admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.027162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a9ea1-ac1a-4d2f-abd9-86d98db1cfff admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.028175 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a9ea1-ac1a-4d2f-abd9-86d98db1cfff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:49.030638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a9ea1-ac1a-4d2f-abd9-86d98db1cfff admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:49.051515 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1032/2072] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:49.055581 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a9ea1-ac1a-4d2f-abd9-86d98db1cfff admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:49.086418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a9ea1-ac1a-4d2f-abd9-86d98db1cfff admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:49.095014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8b3a1f5f92400cb190f7c8b1158961', 'request_id': 'req-7bef6528-0a04-4dc6-8b79-89503abfbffc'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:49.106663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8b3a1f5f92400cb190f7c8b1158961', 'request_id': 'req-7bef6528-0a04-4dc6-8b79-89503abfbffc'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:49.115620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8b3a1f5f92400cb190f7c8b1158961', 'request_id': 'req-7bef6528-0a04-4dc6-8b79-89503abfbffc'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:49.119662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:49.137488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] The token cache is being invalidate because project 6e8b3a1f5f92400cb190f7c8b1158961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:49.137789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:49.139779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:49.143007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1041/2073] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:48 2022] DELETE /identity/v3/projects/6e8b3a1f5f92400cb190f7c8b1158961 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:49.144297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:49.148583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:49.177566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c5b5e-8579-4038-982a-10d93bff0ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:49.178369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c5b5e-8579-4038-982a-10d93bff0ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.178618 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c5b5e-8579-4038-982a-10d93bff0ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.178887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c5b5e-8579-4038-982a-10d93bff0ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.179574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c5b5e-8579-4038-982a-10d93bff0ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:49.181443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c5b5e-8579-4038-982a-10d93bff0ac5 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:49.193772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1033/2074] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:49.195567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc832d07-7bf7-4cc5-b865-0db4406aab20 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.195841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc832d07-7bf7-4cc5-b865-0db4406aab20 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.196075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc832d07-7bf7-4cc5-b865-0db4406aab20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.219917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-fc832d07-7bf7-4cc5-b865-0db4406aab20 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:49.221148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fc832d07-7bf7-4cc5-b865-0db4406aab20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:49.230569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1034/2075] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:49.238509 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc832d07-7bf7-4cc5-b865-0db4406aab20 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:49.239968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fc832d07-7bf7-4cc5-b865-0db4406aab20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:49.252979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc832d07-7bf7-4cc5-b865-0db4406aab20 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:49.259849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22236497-c144-437f-be40-c5e51ee8d86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:49.260763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22236497-c144-437f-be40-c5e51ee8d86c admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.261113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22236497-c144-437f-be40-c5e51ee8d86c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.261497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22236497-c144-437f-be40-c5e51ee8d86c admin admin] PATH_INFO: `/v3/projects/f26c60d5e2e140049abd076b71ba9f20/users/c7693d961dcb45c888089258ddb7c39b/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.280036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22236497-c144-437f-be40-c5e51ee8d86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26c60d5e2e140049abd076b71ba9f20, user_id=c7693d961dcb45c888089258ddb7c39b, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:49.281323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef6528-0a04-4dc6-8b79-89503abfbffc admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:49.283014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22236497-c144-437f-be40-c5e51ee8d86c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:49.294369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0689f4-692f-4306-af79-c327f6857018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:49.295847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0689f4-692f-4306-af79-c327f6857018 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.296285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0689f4-692f-4306-af79-c327f6857018 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.297351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0689f4-692f-4306-af79-c327f6857018 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.298263 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0689f4-692f-4306-af79-c327f6857018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:49.300793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0689f4-692f-4306-af79-c327f6857018 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:49.304410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1035/2076] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:49 2022] PUT /identity/v3/projects/f26c60d5e2e140049abd076b71ba9f20/users/c7693d961dcb45c888089258ddb7c39b/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:49.306427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f16f4-348b-4d58-be7f-10be4208cd00 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.306727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f16f4-348b-4d58-be7f-10be4208cd00 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.306987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f16f4-348b-4d58-be7f-10be4208cd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.323610 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1042/2077] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:49.323610 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0689f4-692f-4306-af79-c327f6857018 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:49.332081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-c61f16f4-348b-4d58-be7f-10be4208cd00 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:49.333531 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0689f4-692f-4306-af79-c327f6857018 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:49.335708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c61f16f4-348b-4d58-be7f-10be4208cd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:49.407736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:49.473846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1036/2078] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:49.474737 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f16f4-348b-4d58-be7f-10be4208cd00 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:49.504421 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f16f4-348b-4d58-be7f-10be4208cd00 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:49.511466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0494c0-1dcf-4cb1-8be1-a0cb0cd1c9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:49.512642 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0494c0-1dcf-4cb1-8be1-a0cb0cd1c9f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.513215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0494c0-1dcf-4cb1-8be1-a0cb0cd1c9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.513512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0494c0-1dcf-4cb1-8be1-a0cb0cd1c9f2 admin admin] PATH_INFO: `/v3/projects/1b868765e302488cb63bd41707517c99/users/bab466138ce24079bc76034833b27a07/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.527597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b68e4-7b5b-41f1-9c20-a394cfc5e3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:49.528652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b68e4-7b5b-41f1-9c20-a394cfc5e3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.529103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b68e4-7b5b-41f1-9c20-a394cfc5e3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.530023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b68e4-7b5b-41f1-9c20-a394cfc5e3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.530836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b68e4-7b5b-41f1-9c20-a394cfc5e3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:49.533641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b68e4-7b5b-41f1-9c20-a394cfc5e3c6 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:49.536791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0494c0-1dcf-4cb1-8be1-a0cb0cd1c9f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b868765e302488cb63bd41707517c99, user_id=bab466138ce24079bc76034833b27a07, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:49.541394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0494c0-1dcf-4cb1-8be1-a0cb0cd1c9f2 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:49.555237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1043/2079] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:49.555237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b68e4-7b5b-41f1-9c20-a394cfc5e3c6 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:49.567581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b68e4-7b5b-41f1-9c20-a394cfc5e3c6 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:49.571236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1037/2080] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:49 2022] PUT /identity/v3/projects/1b868765e302488cb63bd41707517c99/users/bab466138ce24079bc76034833b27a07/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:49.572755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0494c0-1dcf-4cb1-8be1-a0cb0cd1c9f2 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:49.651526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a3ed4-7ca9-480d-a6ce-49f867712766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:49.652146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a3ed4-7ca9-480d-a6ce-49f867712766 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.652428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a3ed4-7ca9-480d-a6ce-49f867712766 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.652702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a3ed4-7ca9-480d-a6ce-49f867712766 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.653371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a3ed4-7ca9-480d-a6ce-49f867712766 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:49.653673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a3ed4-7ca9-480d-a6ce-49f867712766 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:49.655399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a3ed4-7ca9-480d-a6ce-49f867712766 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:49.671793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1044/2081] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:49.673200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f4bfd-7477-4622-91e1-9baa0c063f13 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.673491 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f4bfd-7477-4622-91e1-9baa0c063f13 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.673762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f4bfd-7477-4622-91e1-9baa0c063f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.711662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0494c0-1dcf-4cb1-8be1-a0cb0cd1c9f2 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:49.717590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101487a-0b6c-4d83-9a66-0185ed79739a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:49.718927 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101487a-0b6c-4d83-9a66-0185ed79739a admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.719240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101487a-0b6c-4d83-9a66-0185ed79739a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.719518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101487a-0b6c-4d83-9a66-0185ed79739a admin admin] PATH_INFO: `/v3/projects/f26c60d5e2e140049abd076b71ba9f20/users/c7693d961dcb45c888089258ddb7c39b/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.722810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101487a-0b6c-4d83-9a66-0185ed79739a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26c60d5e2e140049abd076b71ba9f20, user_id=c7693d961dcb45c888089258ddb7c39b, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:49.725676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101487a-0b6c-4d83-9a66-0185ed79739a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:49.727636 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-f31f4bfd-7477-4622-91e1-9baa0c063f13 None None] MFA Rules not processed for user `bab466138ce24079bc76034833b27a07`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:49.729610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:49.730381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-f31f4bfd-7477-4622-91e1-9baa0c063f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:49.751786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1038/2082] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:49 2022] PUT /identity/v3/projects/f26c60d5e2e140049abd076b71ba9f20/users/c7693d961dcb45c888089258ddb7c39b/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:49.754270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101487a-0b6c-4d83-9a66-0185ed79739a admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:49.843855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1045/2083] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:49.845352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efc954-7122-4fa8-bcbd-5588c72aa6e9 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.845742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efc954-7122-4fa8-bcbd-5588c72aa6e9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.846131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efc954-7122-4fa8-bcbd-5588c72aa6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.885614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-11efc954-7122-4fa8-bcbd-5588c72aa6e9 None None] MFA Rules not processed for user `c7693d961dcb45c888089258ddb7c39b`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:49.886856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-11efc954-7122-4fa8-bcbd-5588c72aa6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:49.891655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101487a-0b6c-4d83-9a66-0185ed79739a admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:49.898402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ce3d8-32d4-4705-a099-0f6ee5f44ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:49.899017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ce3d8-32d4-4705-a099-0f6ee5f44ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.899331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ce3d8-32d4-4705-a099-0f6ee5f44ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.899604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ce3d8-32d4-4705-a099-0f6ee5f44ac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.900245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ce3d8-32d4-4705-a099-0f6ee5f44ac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:49.902196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ce3d8-32d4-4705-a099-0f6ee5f44ac3 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:49.926988 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:49.939189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1039/2084] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:49 2022] POST /identity/v3/projects => generated 381 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:49.940469 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ce3d8-32d4-4705-a099-0f6ee5f44ac3 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:49.943273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-428ce3d8-32d4-4705-a099-0f6ee5f44ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:50.007195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1046/2085] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:50.008532 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efc954-7122-4fa8-bcbd-5588c72aa6e9 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:50.035306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efc954-7122-4fa8-bcbd-5588c72aa6e9 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:50.037186 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ce3d8-32d4-4705-a099-0f6ee5f44ac3 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:50.044206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a73d1c-e838-49f6-a5f6-ccae809d9dda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:50.044840 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a73d1c-e838-49f6-a5f6-ccae809d9dda service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:50.047754 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a73d1c-e838-49f6-a5f6-ccae809d9dda service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:50.047910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a73d1c-e838-49f6-a5f6-ccae809d9dda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:50.173356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4cfc7-d326-453d-910e-8c42294e2181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:50.176491 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4cfc7-d326-453d-910e-8c42294e2181 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:50.176907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4cfc7-d326-453d-910e-8c42294e2181 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:50.177283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4cfc7-d326-453d-910e-8c42294e2181 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:50.178243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4cfc7-d326-453d-910e-8c42294e2181 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:50.180187 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a73d1c-e838-49f6-a5f6-ccae809d9dda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:50.180584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4cfc7-d326-453d-910e-8c42294e2181 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:50.183208 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a73d1c-e838-49f6-a5f6-ccae809d9dda service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:50.196730 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1040/2086] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:50.198793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a73d1c-e838-49f6-a5f6-ccae809d9dda service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:50.201881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-02a73d1c-e838-49f6-a5f6-ccae809d9dda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:50.231168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1047/2087] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:50 2022] POST /identity/v3/users => generated 291 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:50.236040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4cfc7-d326-453d-910e-8c42294e2181 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:50.253716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4cfc7-d326-453d-910e-8c42294e2181 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:50.256654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a73d1c-e838-49f6-a5f6-ccae809d9dda service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:50.265581 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175e2cf-f15e-4bd7-a1fd-8eaadc69fc82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:50.266449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175e2cf-f15e-4bd7-a1fd-8eaadc69fc82 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:50.267006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175e2cf-f15e-4bd7-a1fd-8eaadc69fc82 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:50.267542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175e2cf-f15e-4bd7-a1fd-8eaadc69fc82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:50.273250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5175e2cf-f15e-4bd7-a1fd-8eaadc69fc82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:50.295757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c316c10-0521-481d-8346-8c0952866144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:50.297201 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c316c10-0521-481d-8346-8c0952866144 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:50.297997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c316c10-0521-481d-8346-8c0952866144 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:50.299442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c316c10-0521-481d-8346-8c0952866144 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:50.300781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c316c10-0521-481d-8346-8c0952866144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:50.302436 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c316c10-0521-481d-8346-8c0952866144 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:50.324188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1048/2088] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:50.325419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c316c10-0521-481d-8346-8c0952866144 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:50.341866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c316c10-0521-481d-8346-8c0952866144 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:50.366422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:50.369184 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c98445-6932-43a4-a3a6-bcd529837d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:50.371096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c98445-6932-43a4-a3a6-bcd529837d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:50.371734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c98445-6932-43a4-a3a6-bcd529837d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:50.372124 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c98445-6932-43a4-a3a6-bcd529837d89 admin admin] PATH_INFO: `/v3/users/bab466138ce24079bc76034833b27a07` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:50.373737 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c98445-6932-43a4-a3a6-bcd529837d89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bab466138ce24079bc76034833b27a07)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:50.375811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c98445-6932-43a4-a3a6-bcd529837d89 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:50.420961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175e2cf-f15e-4bd7-a1fd-8eaadc69fc82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:50.423076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175e2cf-f15e-4bd7-a1fd-8eaadc69fc82 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:50.436773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1041/2089] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:50.438016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175e2cf-f15e-4bd7-a1fd-8eaadc69fc82 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:50.447783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c8c98445-6932-43a4-a3a6-bcd529837d89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bab466138ce24079bc76034833b27a07'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:50.451908 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175e2cf-f15e-4bd7-a1fd-8eaadc69fc82 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:50.458123 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484d0a9-a63c-4936-baee-d86aa9162f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:50.458744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484d0a9-a63c-4936-baee-d86aa9162f56 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:50.459024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484d0a9-a63c-4936-baee-d86aa9162f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:50.459328 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484d0a9-a63c-4936-baee-d86aa9162f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:50.459987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484d0a9-a63c-4936-baee-d86aa9162f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:50.461681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484d0a9-a63c-4936-baee-d86aa9162f56 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:50.468895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c8c98445-6932-43a4-a3a6-bcd529837d89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bab466138ce24079bc76034833b27a07'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:50.469673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c8c98445-6932-43a4-a3a6-bcd529837d89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bab466138ce24079bc76034833b27a07'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:50.471231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c8c98445-6932-43a4-a3a6-bcd529837d89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bab466138ce24079bc76034833b27a07'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:50.478744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1042/2090] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:50.483264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484d0a9-a63c-4936-baee-d86aa9162f56 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:50.485222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e484d0a9-a63c-4936-baee-d86aa9162f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:50.488968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c8c98445-6932-43a4-a3a6-bcd529837d89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bab466138ce24079bc76034833b27a07'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:50.509279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1049/2091] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:50 2022] DELETE /identity/v3/users/bab466138ce24079bc76034833b27a07 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:50.513935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484d0a9-a63c-4936-baee-d86aa9162f56 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:50.514158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a91e6-294a-4561-973a-dbb60df27427 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:50.514268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a91e6-294a-4561-973a-dbb60df27427 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:50.514495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a91e6-294a-4561-973a-dbb60df27427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:50.534912 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0a0a91e6-294a-4561-973a-dbb60df27427 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:50.536183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0a0a91e6-294a-4561-973a-dbb60df27427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:50.545841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:50.607079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3fb12-d175-418c-a712-ad80d78dfdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:50.608411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3fb12-d175-418c-a712-ad80d78dfdb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:50.608797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3fb12-d175-418c-a712-ad80d78dfdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:50.609342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3fb12-d175-418c-a712-ad80d78dfdb9 admin admin] PATH_INFO: `/v3/projects/18ab895a6071406ea243716457d93503/users/9c67b9c96f3f4837b7c9f6159584d6a6/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:50.611632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1050/2092] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:50.621999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a91e6-294a-4561-973a-dbb60df27427 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:50.623667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0a0a91e6-294a-4561-973a-dbb60df27427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:50.637820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3fb12-d175-418c-a712-ad80d78dfdb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ab895a6071406ea243716457d93503, user_id=9c67b9c96f3f4837b7c9f6159584d6a6, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:50.648805 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3fb12-d175-418c-a712-ad80d78dfdb9 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:50.682181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1043/2093] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:50 2022] PUT /identity/v3/projects/18ab895a6071406ea243716457d93503/users/9c67b9c96f3f4837b7c9f6159584d6a6/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:50.696213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e02a59-8e29-4d6b-8029-5a6f3df38e12 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:50.697213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e02a59-8e29-4d6b-8029-5a6f3df38e12 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:50.697707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e02a59-8e29-4d6b-8029-5a6f3df38e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:50.732503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a91e6-294a-4561-973a-dbb60df27427 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:50.736912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-40e02a59-8e29-4d6b-8029-5a6f3df38e12 None None] MFA Rules not processed for user `9c67b9c96f3f4837b7c9f6159584d6a6`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:50.741557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-40e02a59-8e29-4d6b-8029-5a6f3df38e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:50.823335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00330c6e-ec55-43fc-b94b-61170ada10cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:50.824022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00330c6e-ec55-43fc-b94b-61170ada10cf service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:50.824310 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00330c6e-ec55-43fc-b94b-61170ada10cf service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:50.824859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00330c6e-ec55-43fc-b94b-61170ada10cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:50.867850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1044/2094] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:50.869094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e02a59-8e29-4d6b-8029-5a6f3df38e12 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:50.958566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:50.994384 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00330c6e-ec55-43fc-b94b-61170ada10cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:50.995880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00330c6e-ec55-43fc-b94b-61170ada10cf service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:51.009478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e02a59-8e29-4d6b-8029-5a6f3df38e12 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:51.010234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1051/2095] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:51.020527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f9c6e-568f-44ab-ad32-dfd16252ba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:51.021562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f9c6e-568f-44ab-ad32-dfd16252ba1c admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.022023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f9c6e-568f-44ab-ad32-dfd16252ba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.022491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f9c6e-568f-44ab-ad32-dfd16252ba1c admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.023576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f9c6e-568f-44ab-ad32-dfd16252ba1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:51.027361 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f9c6e-568f-44ab-ad32-dfd16252ba1c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:51.077742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1045/2096] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:50 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:51.082223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00330c6e-ec55-43fc-b94b-61170ada10cf service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:51.093099 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00330c6e-ec55-43fc-b94b-61170ada10cf service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:51.124398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:51.178310 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde397ee-72e5-4680-b16a-203387acc47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:51.179042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde397ee-72e5-4680-b16a-203387acc47d admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.179453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde397ee-72e5-4680-b16a-203387acc47d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.179826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde397ee-72e5-4680-b16a-203387acc47d admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.180555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde397ee-72e5-4680-b16a-203387acc47d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:51.182567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde397ee-72e5-4680-b16a-203387acc47d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:51.222860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1052/2097] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:51 2022] POST /identity/v3/users => generated 297 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:51.226979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f9c6e-568f-44ab-ad32-dfd16252ba1c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:51.238891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f9c6e-568f-44ab-ad32-dfd16252ba1c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:51.244995 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8316eca-7a30-4a0d-8ff9-7bab6eaaa6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:51.245792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8316eca-7a30-4a0d-8ff9-7bab6eaaa6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.246057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8316eca-7a30-4a0d-8ff9-7bab6eaaa6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.246491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8316eca-7a30-4a0d-8ff9-7bab6eaaa6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.247250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8316eca-7a30-4a0d-8ff9-7bab6eaaa6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:51.248943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8316eca-7a30-4a0d-8ff9-7bab6eaaa6f8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:51.263186 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1046/2098] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:51.265877 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde397ee-72e5-4680-b16a-203387acc47d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:51.276227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde397ee-72e5-4680-b16a-203387acc47d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:51.281353 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405caf6-5bf7-4014-9fdc-388c132cc980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:51.282013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405caf6-5bf7-4014-9fdc-388c132cc980 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.282297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405caf6-5bf7-4014-9fdc-388c132cc980 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.282575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405caf6-5bf7-4014-9fdc-388c132cc980 admin admin] PATH_INFO: `/v3/projects/2c6917b0c7a842b4902e69a35effaa66/users/c62cc01a905245f192d1f4d571e84d22/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.299018 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405caf6-5bf7-4014-9fdc-388c132cc980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6917b0c7a842b4902e69a35effaa66, user_id=c62cc01a905245f192d1f4d571e84d22, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:51.303227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405caf6-5bf7-4014-9fdc-388c132cc980 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:51.321348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1053/2099] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:51 2022] PUT /identity/v3/projects/2c6917b0c7a842b4902e69a35effaa66/users/c62cc01a905245f192d1f4d571e84d22/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:51.325008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8316eca-7a30-4a0d-8ff9-7bab6eaaa6f8 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:51.335505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8316eca-7a30-4a0d-8ff9-7bab6eaaa6f8 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:51.364865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405caf6-5bf7-4014-9fdc-388c132cc980 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:51.366750 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9405caf6-5bf7-4014-9fdc-388c132cc980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:51.411327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155e99e-8298-43f3-bc32-461c96647db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:51.411939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155e99e-8298-43f3-bc32-461c96647db2 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.412220 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155e99e-8298-43f3-bc32-461c96647db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.412491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155e99e-8298-43f3-bc32-461c96647db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.413128 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155e99e-8298-43f3-bc32-461c96647db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:51.414745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155e99e-8298-43f3-bc32-461c96647db2 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:51.424803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1047/2100] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:51.430127 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155e99e-8298-43f3-bc32-461c96647db2 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:51.440222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405caf6-5bf7-4014-9fdc-388c132cc980 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:51.441183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155e99e-8298-43f3-bc32-461c96647db2 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:51.445606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b410d6-9f54-4a96-ba9b-9de5c366dae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:51.445921 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b332275-b13d-4102-a087-77ad4e69da9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:51.446240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b410d6-9f54-4a96-ba9b-9de5c366dae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.446520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b410d6-9f54-4a96-ba9b-9de5c366dae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.446672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b332275-b13d-4102-a087-77ad4e69da9a admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.446795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b410d6-9f54-4a96-ba9b-9de5c366dae9 admin admin] PATH_INFO: `/v3/projects/1b868765e302488cb63bd41707517c99` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.446961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b332275-b13d-4102-a087-77ad4e69da9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.447370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b332275-b13d-4102-a087-77ad4e69da9a admin admin] PATH_INFO: `/v3/projects/2c6917b0c7a842b4902e69a35effaa66/users/c62cc01a905245f192d1f4d571e84d22/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.447958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b410d6-9f54-4a96-ba9b-9de5c366dae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b868765e302488cb63bd41707517c99)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:51.449413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b332275-b13d-4102-a087-77ad4e69da9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6917b0c7a842b4902e69a35effaa66, user_id=c62cc01a905245f192d1f4d571e84d22, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:51.449857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b410d6-9f54-4a96-ba9b-9de5c366dae9 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:51.452052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b332275-b13d-4102-a087-77ad4e69da9a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:51.465431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1048/2101] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:51 2022] PUT /identity/v3/projects/2c6917b0c7a842b4902e69a35effaa66/users/c62cc01a905245f192d1f4d571e84d22/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:51.475599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0619b2-bdae-489d-b221-4a642e1d1983 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.475893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0619b2-bdae-489d-b221-4a642e1d1983 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.476147 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0619b2-bdae-489d-b221-4a642e1d1983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.506843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-5d0619b2-bdae-489d-b221-4a642e1d1983 None None] MFA Rules not processed for user `c62cc01a905245f192d1f4d571e84d22`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:51.508169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5d0619b2-bdae-489d-b221-4a642e1d1983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:51.511168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-50b410d6-9f54-4a96-ba9b-9de5c366dae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b868765e302488cb63bd41707517c99', 'request_id': 'req-50b410d6-9f54-4a96-ba9b-9de5c366dae9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:51.522311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-50b410d6-9f54-4a96-ba9b-9de5c366dae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b868765e302488cb63bd41707517c99', 'request_id': 'req-50b410d6-9f54-4a96-ba9b-9de5c366dae9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:51.556216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-50b410d6-9f54-4a96-ba9b-9de5c366dae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b868765e302488cb63bd41707517c99', 'request_id': 'req-50b410d6-9f54-4a96-ba9b-9de5c366dae9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:51.583608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-50b410d6-9f54-4a96-ba9b-9de5c366dae9 admin admin] The token cache is being invalidate because project 1b868765e302488cb63bd41707517c99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:51.584191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-50b410d6-9f54-4a96-ba9b-9de5c366dae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:51.585121 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-50b410d6-9f54-4a96-ba9b-9de5c366dae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:51.588273 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1054/2102] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:51 2022] DELETE /identity/v3/projects/1b868765e302488cb63bd41707517c99 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:51.602146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63131525-aff0-4c7d-ab35-77feb2076d06 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.603565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63131525-aff0-4c7d-ab35-77feb2076d06 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.604399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63131525-aff0-4c7d-ab35-77feb2076d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.631250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1049/2103] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:51.638907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-63131525-aff0-4c7d-ab35-77feb2076d06 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:51.640512 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-63131525-aff0-4c7d-ab35-77feb2076d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:51.646622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0619b2-bdae-489d-b221-4a642e1d1983 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:51.648135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5d0619b2-bdae-489d-b221-4a642e1d1983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:51.672281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:51.679967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:51.767648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1055/2104] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:51.770031 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9d364-e278-460d-b7f5-4a2ebbbb1fac None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.770493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9d364-e278-460d-b7f5-4a2ebbbb1fac None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.770846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9d364-e278-460d-b7f5-4a2ebbbb1fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.791221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0619b2-bdae-489d-b221-4a642e1d1983 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:51.797110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ffa9d364-e278-460d-b7f5-4a2ebbbb1fac None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:51.798975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ffa9d364-e278-460d-b7f5-4a2ebbbb1fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:51.801246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81817e-3d9e-413a-b8f9-041ff7fb4ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:51.802501 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81817e-3d9e-413a-b8f9-041ff7fb4ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.802976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81817e-3d9e-413a-b8f9-041ff7fb4ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.803417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81817e-3d9e-413a-b8f9-041ff7fb4ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.814082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1056/2105] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:51.816764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81817e-3d9e-413a-b8f9-041ff7fb4ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:51.818325 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81817e-3d9e-413a-b8f9-041ff7fb4ea3 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:51.821211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9d364-e278-460d-b7f5-4a2ebbbb1fac None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:51.829117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1050/2106] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:51.834071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9d364-e278-460d-b7f5-4a2ebbbb1fac None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:51.839232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9138e-5630-455e-9dc2-dcc664c9f73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:51.840291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9138e-5630-455e-9dc2-dcc664c9f73a admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.840831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9138e-5630-455e-9dc2-dcc664c9f73a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.841274 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9138e-5630-455e-9dc2-dcc664c9f73a admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.841939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9138e-5630-455e-9dc2-dcc664c9f73a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:51.842358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9138e-5630-455e-9dc2-dcc664c9f73a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:51.843913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9138e-5630-455e-9dc2-dcc664c9f73a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:51.861455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1057/2107] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:51.863758 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81817e-3d9e-413a-b8f9-041ff7fb4ea3 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:51.874555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81817e-3d9e-413a-b8f9-041ff7fb4ea3 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:51.885091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c157f-11e0-4e80-a086-47632408bd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:51.885987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c157f-11e0-4e80-a086-47632408bd52 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:51.886399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c157f-11e0-4e80-a086-47632408bd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:51.886954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c157f-11e0-4e80-a086-47632408bd52 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:51.888152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c157f-11e0-4e80-a086-47632408bd52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:51.892169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c157f-11e0-4e80-a086-47632408bd52 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:51.928832 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1051/2108] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:51 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:51.932868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9138e-5630-455e-9dc2-dcc664c9f73a admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:51.944465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9138e-5630-455e-9dc2-dcc664c9f73a admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:52.022943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a239b06-ced0-4a04-b4ce-d711ed77f727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:52.023606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a239b06-ced0-4a04-b4ce-d711ed77f727 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:52.023925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a239b06-ced0-4a04-b4ce-d711ed77f727 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:52.024199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a239b06-ced0-4a04-b4ce-d711ed77f727 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:52.024845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a239b06-ced0-4a04-b4ce-d711ed77f727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:52.026774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a239b06-ced0-4a04-b4ce-d711ed77f727 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:52.050823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1058/2109] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:51 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:52.055632 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c157f-11e0-4e80-a086-47632408bd52 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:52.068906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c157f-11e0-4e80-a086-47632408bd52 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:52.083354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55c803-e97e-4c7d-b621-cea33f312d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:52.084098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55c803-e97e-4c7d-b621-cea33f312d3d admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:52.084611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55c803-e97e-4c7d-b621-cea33f312d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:52.085181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55c803-e97e-4c7d-b621-cea33f312d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:52.086083 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55c803-e97e-4c7d-b621-cea33f312d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:52.088053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55c803-e97e-4c7d-b621-cea33f312d3d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:52.103356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1052/2110] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:52.113525 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a239b06-ced0-4a04-b4ce-d711ed77f727 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:52.126275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a239b06-ced0-4a04-b4ce-d711ed77f727 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:52.131925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e3603-06be-4cf9-acd4-96570eedf226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:52.132824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e3603-06be-4cf9-acd4-96570eedf226 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:52.133112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e3603-06be-4cf9-acd4-96570eedf226 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:52.133542 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e3603-06be-4cf9-acd4-96570eedf226 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:52.134057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e3603-06be-4cf9-acd4-96570eedf226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:52.136110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e3603-06be-4cf9-acd4-96570eedf226 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:52.149543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1059/2111] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:52.153983 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55c803-e97e-4c7d-b621-cea33f312d3d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:52.166549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55c803-e97e-4c7d-b621-cea33f312d3d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:52.178171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446ab95-9535-4eaa-8320-c81b49620d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:52.179076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446ab95-9535-4eaa-8320-c81b49620d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:52.179466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446ab95-9535-4eaa-8320-c81b49620d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:52.179811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446ab95-9535-4eaa-8320-c81b49620d7c admin admin] PATH_INFO: `/v3/projects/11b846012de64206b9fd987b6c41ecc7/users/7f90e8a8e998441f8955513cbb8fe4b0/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:52.204913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446ab95-9535-4eaa-8320-c81b49620d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=11b846012de64206b9fd987b6c41ecc7, user_id=7f90e8a8e998441f8955513cbb8fe4b0, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:52.208109 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446ab95-9535-4eaa-8320-c81b49620d7c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:52.243162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1053/2112] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:52 2022] PUT /identity/v3/projects/11b846012de64206b9fd987b6c41ecc7/users/7f90e8a8e998441f8955513cbb8fe4b0/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:52.253677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2eda0-d927-4582-b612-730ce13cac9b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:52.254157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2eda0-d927-4582-b612-730ce13cac9b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:52.254318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446ab95-9535-4eaa-8320-c81b49620d7c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:52.254491 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2eda0-d927-4582-b612-730ce13cac9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:52.314254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-f2f2eda0-d927-4582-b612-730ce13cac9b None None] MFA Rules not processed for user `7f90e8a8e998441f8955513cbb8fe4b0`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:52.317104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-f2f2eda0-d927-4582-b612-730ce13cac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:52.375914 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446ab95-9535-4eaa-8320-c81b49620d7c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:52.382030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b89b6-696a-422d-83be-a628e64893b0 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62cc01a905245f192d1f4d571e84d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c6917b0c7a842b4902e69a35effaa66', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:52.382952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b89b6-696a-422d-83be-a628e64893b0 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:52.383479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b89b6-696a-422d-83be-a628e64893b0 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:52.384088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b89b6-696a-422d-83be-a628e64893b0 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:52.384900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b89b6-696a-422d-83be-a628e64893b0 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:52.385454 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b89b6-696a-422d-83be-a628e64893b0 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:52.387326 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b89b6-696a-422d-83be-a628e64893b0 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:52.405177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1054/2113] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:49:52.407167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b89b6-696a-422d-83be-a628e64893b0 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:52.425022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1060/2114] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:52.430572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2eda0-d927-4582-b612-730ce13cac9b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:52.443332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2eda0-d927-4582-b612-730ce13cac9b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:52.448271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fa90b-de85-43aa-8742-4d2c32f094f1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62cc01a905245f192d1f4d571e84d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c6917b0c7a842b4902e69a35effaa66', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:52.449938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fa90b-de85-43aa-8742-4d2c32f094f1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:52.449938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fa90b-de85-43aa-8742-4d2c32f094f1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:52.449938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fa90b-de85-43aa-8742-4d2c32f094f1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:52.450490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68fa90b-de85-43aa-8742-4d2c32f094f1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:52.452633 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68fa90b-de85-43aa-8742-4d2c32f094f1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:52.459286 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:52.491673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b89b6-696a-422d-83be-a628e64893b0 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:52.492741 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1061/2115] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:52 2022] POST /identity/v3/projects => generated 354 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:52.496216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fa90b-de85-43aa-8742-4d2c32f094f1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:52.498229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e68fa90b-de85-43aa-8742-4d2c32f094f1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:52.503875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:52.553211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69b0d5-657a-4978-8e66-94e59e3fd8c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:52.553986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69b0d5-657a-4978-8e66-94e59e3fd8c0 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:52.554318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69b0d5-657a-4978-8e66-94e59e3fd8c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:52.554670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69b0d5-657a-4978-8e66-94e59e3fd8c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:52.556744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7f69b0d5-657a-4978-8e66-94e59e3fd8c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:52.572682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fa90b-de85-43aa-8742-4d2c32f094f1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:52.578777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937443ce-36b9-413e-b2af-e2d89c790cef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:52.579848 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937443ce-36b9-413e-b2af-e2d89c790cef service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:52.580339 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937443ce-36b9-413e-b2af-e2d89c790cef service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:52.580816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937443ce-36b9-413e-b2af-e2d89c790cef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:52.623558 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69b0d5-657a-4978-8e66-94e59e3fd8c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:52.625158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69b0d5-657a-4978-8e66-94e59e3fd8c0 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:52.635493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1055/2116] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:52.636334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69b0d5-657a-4978-8e66-94e59e3fd8c0 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:52.668253 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937443ce-36b9-413e-b2af-e2d89c790cef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:52.669715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937443ce-36b9-413e-b2af-e2d89c790cef service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:52.680307 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1062/2117] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:52.709533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69b0d5-657a-4978-8e66-94e59e3fd8c0 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:52.717481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5d93d-71d0-4e5a-a279-6c2452264844 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62cc01a905245f192d1f4d571e84d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c6917b0c7a842b4902e69a35effaa66', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:52.718111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5d93d-71d0-4e5a-a279-6c2452264844 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:52.718814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5d93d-71d0-4e5a-a279-6c2452264844 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:52.719520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5d93d-71d0-4e5a-a279-6c2452264844 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:52.723237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5d93d-71d0-4e5a-a279-6c2452264844 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:52.723237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5d93d-71d0-4e5a-a279-6c2452264844 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:52.723594 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5d93d-71d0-4e5a-a279-6c2452264844 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:52.737434 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1056/2118] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:53.130772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937443ce-36b9-413e-b2af-e2d89c790cef service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:53.158987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937443ce-36b9-413e-b2af-e2d89c790cef service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:53.172939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62cc01a905245f192d1f4d571e84d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c6917b0c7a842b4902e69a35effaa66', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:53.175940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:53.176751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:53.177204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] PATH_INFO: `/v3/projects/422374b5f9214ce28942a13c4ab9635e` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:53.179442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: Authorizing `identity:delete_project(project_id=422374b5f9214ce28942a13c4ab9635e)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:53.185153 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:53.275404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '422374b5f9214ce28942a13c4ab9635e', 'request_id': 'req-5d8cebda-bbba-430d-920d-ba995addbea1'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:53.286386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '422374b5f9214ce28942a13c4ab9635e', 'request_id': 'req-5d8cebda-bbba-430d-920d-ba995addbea1'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:53.296421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '422374b5f9214ce28942a13c4ab9635e', 'request_id': 'req-5d8cebda-bbba-430d-920d-ba995addbea1'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:53.318499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] The token cache is being invalidate because project 422374b5f9214ce28942a13c4ab9635e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:53.318990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:53.319744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:53.324935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1063/2119] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:53 2022] DELETE /identity/v3/projects/422374b5f9214ce28942a13c4ab9635e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:53.444965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5d93d-71d0-4e5a-a279-6c2452264844 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:53.447013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e9d5d93d-71d0-4e5a-a279-6c2452264844 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:53.461239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5d93d-71d0-4e5a-a279-6c2452264844 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:53.520317 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703bdbe-3d24-45da-9596-eb207c8aaffd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:53.520948 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703bdbe-3d24-45da-9596-eb207c8aaffd service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:53.521242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703bdbe-3d24-45da-9596-eb207c8aaffd service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:53.521513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703bdbe-3d24-45da-9596-eb207c8aaffd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:53.523293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b703bdbe-3d24-45da-9596-eb207c8aaffd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:53.537559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:53.569858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:53.571828 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:53.595694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703bdbe-3d24-45da-9596-eb207c8aaffd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:53.597779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703bdbe-3d24-45da-9596-eb207c8aaffd service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:53.600737 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:53.608279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1057/2120] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:53.645514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cebda-bbba-430d-920d-ba995addbea1 tempest-VolumeQuotaClassesTest-1574875040 tempest-VolumeQuotaClassesTest-1574875040-project-admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:53.651477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:53.652145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:53.652424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:53.652698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] PATH_INFO: `/v3/users/9c67b9c96f3f4837b7c9f6159584d6a6` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:53.653840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c67b9c96f3f4837b7c9f6159584d6a6)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:53.655876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:53.709730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c67b9c96f3f4837b7c9f6159584d6a6'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:53.723138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c67b9c96f3f4837b7c9f6159584d6a6'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:53.723855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c67b9c96f3f4837b7c9f6159584d6a6'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:53.724483 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c67b9c96f3f4837b7c9f6159584d6a6'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:53.732027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c67b9c96f3f4837b7c9f6159584d6a6'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:53.745902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1064/2121] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:53 2022] DELETE /identity/v3/users/9c67b9c96f3f4837b7c9f6159584d6a6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:53.750551 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffff6e1-9575-47d5-a878-b24be9aade98 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:53.750837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffff6e1-9575-47d5-a878-b24be9aade98 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:53.751111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffff6e1-9575-47d5-a878-b24be9aade98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:53.774789 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-6ffff6e1-9575-47d5-a878-b24be9aade98 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:53.775963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6ffff6e1-9575-47d5-a878-b24be9aade98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:53.837326 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1058/2122] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:53.847992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:53.849563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:53.916704 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91bb81b-68db-4aff-ac51-e2d354572355 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:53.920461 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ed53d-51f4-4c6c-ad73-f74861d03fab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:53.921074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ed53d-51f4-4c6c-ad73-f74861d03fab service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:53.921357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ed53d-51f4-4c6c-ad73-f74861d03fab service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:53.921633 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ed53d-51f4-4c6c-ad73-f74861d03fab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:53.931824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ed53d-51f4-4c6c-ad73-f74861d03fab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:53.933274 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ed53d-51f4-4c6c-ad73-f74861d03fab service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:53.944139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1065/2123] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:54.328533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffff6e1-9575-47d5-a878-b24be9aade98 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:54.330350 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6ffff6e1-9575-47d5-a878-b24be9aade98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:54.347188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffff6e1-9575-47d5-a878-b24be9aade98 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:54.354860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:54.355687 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:54.356096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:54.356416 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] PATH_INFO: `/v3/projects/18ab895a6071406ea243716457d93503` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:54.357918 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] RBAC: Authorizing `identity:delete_project(project_id=18ab895a6071406ea243716457d93503)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:54.360295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:54.433580 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18ab895a6071406ea243716457d93503', 'request_id': 'req-68684005-94b1-48d9-980b-7abfc612486f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:54.442744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18ab895a6071406ea243716457d93503', 'request_id': 'req-68684005-94b1-48d9-980b-7abfc612486f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:54.449265 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18ab895a6071406ea243716457d93503', 'request_id': 'req-68684005-94b1-48d9-980b-7abfc612486f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:54.488847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] The token cache is being invalidate because project 18ab895a6071406ea243716457d93503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:54.489086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:54.490014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:54.496766 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1059/2124] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:54 2022] DELETE /identity/v3/projects/18ab895a6071406ea243716457d93503 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:54.504112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ed53d-51f4-4c6c-ad73-f74861d03fab service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:54.528213 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ed53d-51f4-4c6c-ad73-f74861d03fab service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:54.609019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:54.637070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e12992-420f-471c-a372-f9e007a3c6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:54.637726 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e12992-420f-471c-a372-f9e007a3c6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:54.638010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e12992-420f-471c-a372-f9e007a3c6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:54.638281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e12992-420f-471c-a372-f9e007a3c6ee admin admin] PATH_INFO: `/v3/users/c62cc01a905245f192d1f4d571e84d22` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:54.639597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e12992-420f-471c-a372-f9e007a3c6ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c62cc01a905245f192d1f4d571e84d22)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:54.641631 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e12992-420f-471c-a372-f9e007a3c6ee admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:54.717284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-32e12992-420f-471c-a372-f9e007a3c6ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c62cc01a905245f192d1f4d571e84d22'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:54.738582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-32e12992-420f-471c-a372-f9e007a3c6ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c62cc01a905245f192d1f4d571e84d22'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:54.739833 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-32e12992-420f-471c-a372-f9e007a3c6ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c62cc01a905245f192d1f4d571e84d22'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:54.740860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-32e12992-420f-471c-a372-f9e007a3c6ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c62cc01a905245f192d1f4d571e84d22'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:54.755578 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-32e12992-420f-471c-a372-f9e007a3c6ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c62cc01a905245f192d1f4d571e84d22'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:54.771202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1066/2125] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:54 2022] DELETE /identity/v3/users/c62cc01a905245f192d1f4d571e84d22 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:55.271260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:55.274792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:55.310487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68684005-94b1-48d9-980b-7abfc612486f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:55.380609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:55.416468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7b9d9-1e03-47d8-9916-553bd41906e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:55.417272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7b9d9-1e03-47d8-9916-553bd41906e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:55.417623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7b9d9-1e03-47d8-9916-553bd41906e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:55.417954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7b9d9-1e03-47d8-9916-553bd41906e8 admin admin] PATH_INFO: `/v3/projects/2c6917b0c7a842b4902e69a35effaa66` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:55.419372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7b9d9-1e03-47d8-9916-553bd41906e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c6917b0c7a842b4902e69a35effaa66)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:55.421573 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7b9d9-1e03-47d8-9916-553bd41906e8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:55.499803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-46e7b9d9-1e03-47d8-9916-553bd41906e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c6917b0c7a842b4902e69a35effaa66', 'request_id': 'req-46e7b9d9-1e03-47d8-9916-553bd41906e8'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:55.508355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-46e7b9d9-1e03-47d8-9916-553bd41906e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c6917b0c7a842b4902e69a35effaa66', 'request_id': 'req-46e7b9d9-1e03-47d8-9916-553bd41906e8'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:55.515428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-46e7b9d9-1e03-47d8-9916-553bd41906e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c6917b0c7a842b4902e69a35effaa66', 'request_id': 'req-46e7b9d9-1e03-47d8-9916-553bd41906e8'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:55.544617 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-46e7b9d9-1e03-47d8-9916-553bd41906e8 admin admin] The token cache is being invalidate because project 2c6917b0c7a842b4902e69a35effaa66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:55.545884 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-46e7b9d9-1e03-47d8-9916-553bd41906e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:55.547347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-46e7b9d9-1e03-47d8-9916-553bd41906e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:55.551434 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1060/2126] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:55 2022] DELETE /identity/v3/projects/2c6917b0c7a842b4902e69a35effaa66 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:55.558268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99445586-2322-4a21-8407-6a537cd0c2c8 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:55.559145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99445586-2322-4a21-8407-6a537cd0c2c8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:55.559830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99445586-2322-4a21-8407-6a537cd0c2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:55.598899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-99445586-2322-4a21-8407-6a537cd0c2c8 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:55.600576 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-99445586-2322-4a21-8407-6a537cd0c2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:55.706779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1067/2127] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:55.719955 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39033273-1e7d-4b4e-b46a-7108652cd626 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:55.720468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39033273-1e7d-4b4e-b46a-7108652cd626 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:55.721481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39033273-1e7d-4b4e-b46a-7108652cd626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:55.744646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-39033273-1e7d-4b4e-b46a-7108652cd626 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:55.745859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-39033273-1e7d-4b4e-b46a-7108652cd626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:55.754945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1061/2128] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:55.759845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99445586-2322-4a21-8407-6a537cd0c2c8 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:55.770778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99445586-2322-4a21-8407-6a537cd0c2c8 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:55.777172 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f45f94-4465-410a-879e-619fc61ac2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:55.777783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f45f94-4465-410a-879e-619fc61ac2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:55.778237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f45f94-4465-410a-879e-619fc61ac2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:55.778509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f45f94-4465-410a-879e-619fc61ac2e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:55.779219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f45f94-4465-410a-879e-619fc61ac2e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:55.779527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f45f94-4465-410a-879e-619fc61ac2e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:55.781150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f45f94-4465-410a-879e-619fc61ac2e5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:55.797232 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1068/2129] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:55.798841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39033273-1e7d-4b4e-b46a-7108652cd626 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:55.812942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39033273-1e7d-4b4e-b46a-7108652cd626 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:55.820243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495e915-1ead-4e74-980e-37c6ce4842ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:55.821249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495e915-1ead-4e74-980e-37c6ce4842ea admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:55.821529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495e915-1ead-4e74-980e-37c6ce4842ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:55.822026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495e915-1ead-4e74-980e-37c6ce4842ea admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:55.822742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495e915-1ead-4e74-980e-37c6ce4842ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:55.825814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495e915-1ead-4e74-980e-37c6ce4842ea admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:55.861129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1062/2130] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:55 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:55.865599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f45f94-4465-410a-879e-619fc61ac2e5 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:55.876113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f45f94-4465-410a-879e-619fc61ac2e5 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:55.931472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:55.946176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04002304-d665-4e4b-bee7-d31899d14a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:55.946991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04002304-d665-4e4b-bee7-d31899d14a9a admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:55.947516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04002304-d665-4e4b-bee7-d31899d14a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:55.948545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04002304-d665-4e4b-bee7-d31899d14a9a admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:55.948545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04002304-d665-4e4b-bee7-d31899d14a9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:55.950724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04002304-d665-4e4b-bee7-d31899d14a9a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:55.979769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1069/2131] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:55 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:55.984437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495e915-1ead-4e74-980e-37c6ce4842ea admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:56.000673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495e915-1ead-4e74-980e-37c6ce4842ea admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:56.009668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65b3c0-76aa-42c1-af4c-5a7fd007d2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:56.010747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65b3c0-76aa-42c1-af4c-5a7fd007d2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:56.011279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65b3c0-76aa-42c1-af4c-5a7fd007d2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:56.011774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65b3c0-76aa-42c1-af4c-5a7fd007d2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:56.012969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65b3c0-76aa-42c1-af4c-5a7fd007d2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:56.016185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65b3c0-76aa-42c1-af4c-5a7fd007d2b1 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:56.036490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1063/2132] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:56.041873 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04002304-d665-4e4b-bee7-d31899d14a9a admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:56.059476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04002304-d665-4e4b-bee7-d31899d14a9a admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:56.069621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96f173-28c1-4218-8f7c-f5b5b4dba72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:56.070700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96f173-28c1-4218-8f7c-f5b5b4dba72f admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:56.071240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96f173-28c1-4218-8f7c-f5b5b4dba72f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:56.071952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96f173-28c1-4218-8f7c-f5b5b4dba72f admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:56.073674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96f173-28c1-4218-8f7c-f5b5b4dba72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:56.076944 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96f173-28c1-4218-8f7c-f5b5b4dba72f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:56.090437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1070/2133] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:56.095185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65b3c0-76aa-42c1-af4c-5a7fd007d2b1 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:56.108940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65b3c0-76aa-42c1-af4c-5a7fd007d2b1 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:56.114348 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ef61c-3ba9-4959-87e1-4d37771dc873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:56.118486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef61c-3ba9-4959-87e1-4d37771dc873 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:56.119016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef61c-3ba9-4959-87e1-4d37771dc873 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:56.119425 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef61c-3ba9-4959-87e1-4d37771dc873 admin admin] PATH_INFO: `/v3/projects/92695c2be4a94cb1ac423bc75b0afb0a/users/216fd1470e6c44c3a8f88f9eb4c46d3f/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:56.141740 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ef61c-3ba9-4959-87e1-4d37771dc873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92695c2be4a94cb1ac423bc75b0afb0a, user_id=216fd1470e6c44c3a8f88f9eb4c46d3f, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:56.145653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ef61c-3ba9-4959-87e1-4d37771dc873 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:56.166415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1064/2134] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:56 2022] PUT /identity/v3/projects/92695c2be4a94cb1ac423bc75b0afb0a/users/216fd1470e6c44c3a8f88f9eb4c46d3f/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:56.180234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f37ec7-1525-41b0-891e-87d578111305 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:56.181114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f37ec7-1525-41b0-891e-87d578111305 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:56.181543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f37ec7-1525-41b0-891e-87d578111305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:56.232561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-24f37ec7-1525-41b0-891e-87d578111305 None None] MFA Rules not processed for user `216fd1470e6c44c3a8f88f9eb4c46d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:56.233890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-24f37ec7-1525-41b0-891e-87d578111305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:56.332670 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1071/2135] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:56.335563 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ef61c-3ba9-4959-87e1-4d37771dc873 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:56.442934 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ef61c-3ba9-4959-87e1-4d37771dc873 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:56.448898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d694d95-8451-4d4f-8a77-946aa5e2809b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:56.449673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d694d95-8451-4d4f-8a77-946aa5e2809b admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:56.449948 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d694d95-8451-4d4f-8a77-946aa5e2809b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:56.450243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d694d95-8451-4d4f-8a77-946aa5e2809b admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:56.450912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d694d95-8451-4d4f-8a77-946aa5e2809b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:56.452864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d694d95-8451-4d4f-8a77-946aa5e2809b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:56.464136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:56.491168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1065/2136] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:56 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:56.495815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f37ec7-1525-41b0-891e-87d578111305 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:56.509383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f37ec7-1525-41b0-891e-87d578111305 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:56.584045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efe0a5-9c8f-4ae9-b70a-93ce3fa6da5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:56.584996 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efe0a5-9c8f-4ae9-b70a-93ce3fa6da5c admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:56.585315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efe0a5-9c8f-4ae9-b70a-93ce3fa6da5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:56.585556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efe0a5-9c8f-4ae9-b70a-93ce3fa6da5c admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:56.586229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3efe0a5-9c8f-4ae9-b70a-93ce3fa6da5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:56.588665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3efe0a5-9c8f-4ae9-b70a-93ce3fa6da5c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:56.617168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1072/2137] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:56 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:56.619289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d694d95-8451-4d4f-8a77-946aa5e2809b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:56.637643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d694d95-8451-4d4f-8a77-946aa5e2809b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:56.648818 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe8cf3-4e29-4e48-9709-c05924ff0a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:56.649471 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe8cf3-4e29-4e48-9709-c05924ff0a20 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:56.649831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe8cf3-4e29-4e48-9709-c05924ff0a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:56.650202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe8cf3-4e29-4e48-9709-c05924ff0a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:56.651069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe8cf3-4e29-4e48-9709-c05924ff0a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:56.652853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe8cf3-4e29-4e48-9709-c05924ff0a20 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:56.673487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1066/2138] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:56.674752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efe0a5-9c8f-4ae9-b70a-93ce3fa6da5c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:56.688311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efe0a5-9c8f-4ae9-b70a-93ce3fa6da5c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:56.699655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed86697-eb7e-4301-8eff-7aa6f67b430a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:56.701838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed86697-eb7e-4301-8eff-7aa6f67b430a admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:56.702314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed86697-eb7e-4301-8eff-7aa6f67b430a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:56.702767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed86697-eb7e-4301-8eff-7aa6f67b430a admin admin] PATH_INFO: `/v3/projects/327a2a341d764d1bbac913461546b4c3/users/2007a37f1b0c4e748f8b1da3874236c9/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:56.730283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed86697-eb7e-4301-8eff-7aa6f67b430a admin admin] RBAC: Authorizing `identity:create_grant(project_id=327a2a341d764d1bbac913461546b4c3, user_id=2007a37f1b0c4e748f8b1da3874236c9, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:56.735422 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed86697-eb7e-4301-8eff-7aa6f67b430a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:56.756703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1073/2139] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:56 2022] PUT /identity/v3/projects/327a2a341d764d1bbac913461546b4c3/users/2007a37f1b0c4e748f8b1da3874236c9/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:56.760981 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe8cf3-4e29-4e48-9709-c05924ff0a20 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:56.776426 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe8cf3-4e29-4e48-9709-c05924ff0a20 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:56.859874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485afae2-5efb-4aaa-a4e9-a6af7137e0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:56.860946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485afae2-5efb-4aaa-a4e9-a6af7137e0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:56.861382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485afae2-5efb-4aaa-a4e9-a6af7137e0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:56.861659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485afae2-5efb-4aaa-a4e9-a6af7137e0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:56.863069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485afae2-5efb-4aaa-a4e9-a6af7137e0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:56.865058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485afae2-5efb-4aaa-a4e9-a6af7137e0e8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:56.880015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1067/2140] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:56.884103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed86697-eb7e-4301-8eff-7aa6f67b430a admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:56.896802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed86697-eb7e-4301-8eff-7aa6f67b430a admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:56.906698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14a1cf-745a-4872-bde5-024bfb92f7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:56.907564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14a1cf-745a-4872-bde5-024bfb92f7b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:56.907957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14a1cf-745a-4872-bde5-024bfb92f7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:56.908325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14a1cf-745a-4872-bde5-024bfb92f7b6 admin admin] PATH_INFO: `/v3/projects/327a2a341d764d1bbac913461546b4c3/users/2007a37f1b0c4e748f8b1da3874236c9/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:56.910722 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14a1cf-745a-4872-bde5-024bfb92f7b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=327a2a341d764d1bbac913461546b4c3, user_id=2007a37f1b0c4e748f8b1da3874236c9, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:56.913753 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14a1cf-745a-4872-bde5-024bfb92f7b6 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:56.928184 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1074/2141] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:56 2022] PUT /identity/v3/projects/327a2a341d764d1bbac913461546b4c3/users/2007a37f1b0c4e748f8b1da3874236c9/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:56.939285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b90a9-453f-4dc6-818e-9e0b36e18266 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:56.939621 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b90a9-453f-4dc6-818e-9e0b36e18266 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:56.939831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b90a9-453f-4dc6-818e-9e0b36e18266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:56.986115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-8b4b90a9-453f-4dc6-818e-9e0b36e18266 None None] MFA Rules not processed for user `2007a37f1b0c4e748f8b1da3874236c9`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:56.988037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-8b4b90a9-453f-4dc6-818e-9e0b36e18266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:57.095756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1068/2142] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:57.114773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14a1cf-745a-4872-bde5-024bfb92f7b6 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:57.118109 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-5e14a1cf-745a-4872-bde5-024bfb92f7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:57.140488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:57.199372 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14a1cf-745a-4872-bde5-024bfb92f7b6 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:57.203908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9f94f-e709-47ee-aee5-d08bb21a9abb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:57.204810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9f94f-e709-47ee-aee5-d08bb21a9abb service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:57.205049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9f94f-e709-47ee-aee5-d08bb21a9abb service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:57.205329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9f94f-e709-47ee-aee5-d08bb21a9abb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:57.217789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9f94f-e709-47ee-aee5-d08bb21a9abb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:57.220669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9f94f-e709-47ee-aee5-d08bb21a9abb service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:57.230430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1075/2143] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:57.365741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b90a9-453f-4dc6-818e-9e0b36e18266 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:57.376876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b90a9-453f-4dc6-818e-9e0b36e18266 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:57.383673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c74274-b702-4640-8c69-355dff707ea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:57.384561 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74274-b702-4640-8c69-355dff707ea9 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:57.385051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74274-b702-4640-8c69-355dff707ea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:57.385520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74274-b702-4640-8c69-355dff707ea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:57.461891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:57.464858 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c74274-b702-4640-8c69-355dff707ea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:57.466743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c74274-b702-4640-8c69-355dff707ea9 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:57.479751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1069/2144] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:57.840319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9f94f-e709-47ee-aee5-d08bb21a9abb service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:57.937467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9f94f-e709-47ee-aee5-d08bb21a9abb service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:57.948429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:57.949334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:57.949784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:57.950366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] PATH_INFO: `/v3/users/216fd1470e6c44c3a8f88f9eb4c46d3f` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:57.952519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=216fd1470e6c44c3a8f88f9eb4c46d3f)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:57.956709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:58.029170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216fd1470e6c44c3a8f88f9eb4c46d3f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:58.043913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216fd1470e6c44c3a8f88f9eb4c46d3f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:58.044677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216fd1470e6c44c3a8f88f9eb4c46d3f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:58.045392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216fd1470e6c44c3a8f88f9eb4c46d3f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:58.056758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216fd1470e6c44c3a8f88f9eb4c46d3f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:58.085730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1076/2145] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:57 2022] DELETE /identity/v3/users/216fd1470e6c44c3a8f88f9eb4c46d3f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:58.091876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86e592-9225-4946-a646-fe55b0bdd2d3 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:58.093153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86e592-9225-4946-a646-fe55b0bdd2d3 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:58.093629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86e592-9225-4946-a646-fe55b0bdd2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:58.124154 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4c86e592-9225-4946-a646-fe55b0bdd2d3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:58.125640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4c86e592-9225-4946-a646-fe55b0bdd2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:58.213849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1070/2146] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:58.224306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:58.225912 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:58.270360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:58.295340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585c541-554c-41d9-a0fe-7fa9bf1a7dc1 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:58.299258 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84551cb-15bb-4757-8288-e53370d047f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:58.299879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84551cb-15bb-4757-8288-e53370d047f7 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:58.300154 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84551cb-15bb-4757-8288-e53370d047f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:58.300660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84551cb-15bb-4757-8288-e53370d047f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:58.310385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84551cb-15bb-4757-8288-e53370d047f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:58.311868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84551cb-15bb-4757-8288-e53370d047f7 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:58.322331 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1077/2147] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:59.271781 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86e592-9225-4946-a646-fe55b0bdd2d3 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:59.273294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4c86e592-9225-4946-a646-fe55b0bdd2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:59.297837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86e592-9225-4946-a646-fe55b0bdd2d3 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:59.305531 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:59.307008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:59.307582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:59.308000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] PATH_INFO: `/v3/projects/92695c2be4a94cb1ac423bc75b0afb0a` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:59.310320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] RBAC: Authorizing `identity:delete_project(project_id=92695c2be4a94cb1ac423bc75b0afb0a)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:59.312511 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:59.392831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92695c2be4a94cb1ac423bc75b0afb0a', 'request_id': 'req-f8f51518-9518-475a-8730-c3846057b34e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:59.410114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92695c2be4a94cb1ac423bc75b0afb0a', 'request_id': 'req-f8f51518-9518-475a-8730-c3846057b34e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:59.419733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92695c2be4a94cb1ac423bc75b0afb0a', 'request_id': 'req-f8f51518-9518-475a-8730-c3846057b34e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:59.448226 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] The token cache is being invalidate because project 92695c2be4a94cb1ac423bc75b0afb0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:59.448524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:59.449383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:59.452615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1071/2148] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:59 2022] DELETE /identity/v3/projects/92695c2be4a94cb1ac423bc75b0afb0a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:59.459633 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84551cb-15bb-4757-8288-e53370d047f7 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:59.472375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84551cb-15bb-4757-8288-e53370d047f7 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:59.550526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:59.551209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:59.551494 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:59.551766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] PATH_INFO: `/v3/users/2007a37f1b0c4e748f8b1da3874236c9` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:59.552974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2007a37f1b0c4e748f8b1da3874236c9)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:59.555061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:59.610631 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2007a37f1b0c4e748f8b1da3874236c9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:59.626899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2007a37f1b0c4e748f8b1da3874236c9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:59.630554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2007a37f1b0c4e748f8b1da3874236c9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:59.630554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2007a37f1b0c4e748f8b1da3874236c9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:59.638868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2007a37f1b0c4e748f8b1da3874236c9'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:59.657149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1078/2149] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:59 2022] DELETE /identity/v3/users/2007a37f1b0c4e748f8b1da3874236c9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:59.812763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:59.814474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:59.836996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f51518-9518-475a-8730-c3846057b34e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:59.927274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:59.927960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:59.928211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:59.928546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] PATH_INFO: `/v3/users/8a3df25e740c4c7f8a311d5a1574c3ba` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:59.929810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a3df25e740c4c7f8a311d5a1574c3ba)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:59.932195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:59.969029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:00.001549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a3df25e740c4c7f8a311d5a1574c3ba'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.012751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a3df25e740c4c7f8a311d5a1574c3ba'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.013466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a3df25e740c4c7f8a311d5a1574c3ba'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.030674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a3df25e740c4c7f8a311d5a1574c3ba'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.031512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a3df25e740c4c7f8a311d5a1574c3ba'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.047332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1072/2150] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:59 2022] DELETE /identity/v3/users/8a3df25e740c4c7f8a311d5a1574c3ba => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:00.099301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:00.101032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:00.156954 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515361e-91f9-4e54-b1b7-d3e77a7e71b4 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:00.254065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:00.264609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:00.265193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.265999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.266653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] PATH_INFO: `/v3/projects/327a2a341d764d1bbac913461546b4c3` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.269840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=327a2a341d764d1bbac913461546b4c3)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:00.271815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:00.287053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:00.288731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:00.309274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab852a2b-8cea-489b-bc78-cab65e35f112 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:00.314460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68488361-39b0-4f3a-ac5a-bd71f250db7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:00.315372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68488361-39b0-4f3a-ac5a-bd71f250db7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.315588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68488361-39b0-4f3a-ac5a-bd71f250db7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.315861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68488361-39b0-4f3a-ac5a-bd71f250db7c admin admin] PATH_INFO: `/v3/users/290debe72e0a4630ae3e1481a8f24453` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.317168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68488361-39b0-4f3a-ac5a-bd71f250db7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=290debe72e0a4630ae3e1481a8f24453)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:00.319481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68488361-39b0-4f3a-ac5a-bd71f250db7c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:00.345202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '327a2a341d764d1bbac913461546b4c3', 'request_id': 'req-87aeea24-6a60-4063-bf25-d9c367572b26'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.356472 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '327a2a341d764d1bbac913461546b4c3', 'request_id': 'req-87aeea24-6a60-4063-bf25-d9c367572b26'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.365257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '327a2a341d764d1bbac913461546b4c3', 'request_id': 'req-87aeea24-6a60-4063-bf25-d9c367572b26'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.381556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] The token cache is being invalidate because project 327a2a341d764d1bbac913461546b4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:00.381943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.382675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.384966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1079/2151] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:00 2022] DELETE /identity/v3/projects/327a2a341d764d1bbac913461546b4c3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:00.386679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:00.389199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:00.394363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68488361-39b0-4f3a-ac5a-bd71f250db7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '290debe72e0a4630ae3e1481a8f24453'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.408088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68488361-39b0-4f3a-ac5a-bd71f250db7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '290debe72e0a4630ae3e1481a8f24453'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.409843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68488361-39b0-4f3a-ac5a-bd71f250db7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '290debe72e0a4630ae3e1481a8f24453'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.426836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68488361-39b0-4f3a-ac5a-bd71f250db7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '290debe72e0a4630ae3e1481a8f24453'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.427844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68488361-39b0-4f3a-ac5a-bd71f250db7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '290debe72e0a4630ae3e1481a8f24453'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.443611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1073/2152] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:00 2022] DELETE /identity/v3/users/290debe72e0a4630ae3e1481a8f24453 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:00.445117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324e5c0-bcc7-4342-8e07-acd4ecdd88b6 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.445553 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324e5c0-bcc7-4342-8e07-acd4ecdd88b6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.445947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324e5c0-bcc7-4342-8e07-acd4ecdd88b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.470166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-d324e5c0-bcc7-4342-8e07-acd4ecdd88b6 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:00.471679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d324e5c0-bcc7-4342-8e07-acd4ecdd88b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:00.475941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aeea24-6a60-4063-bf25-d9c367572b26 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:00.538366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:00.538921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.539235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.539513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] PATH_INFO: `/v3/projects/09f25e83fff24a8e924bb22e6f9fdd46` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.550200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1074/2153] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:00.550410 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09f25e83fff24a8e924bb22e6f9fdd46)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:00.550507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0eb94-e464-4dc9-98c0-8e225530afe5 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.550507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0eb94-e464-4dc9-98c0-8e225530afe5 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.550507 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0eb94-e464-4dc9-98c0-8e225530afe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.550753 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:00.566241 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-93a0eb94-e464-4dc9-98c0-8e225530afe5 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:00.567559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-93a0eb94-e464-4dc9-98c0-8e225530afe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:00.579668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1075/2154] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:00.579668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3d53f-a34a-4a89-8f04-81804be6f759 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.579668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3d53f-a34a-4a89-8f04-81804be6f759 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.579668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3d53f-a34a-4a89-8f04-81804be6f759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.592670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:00.610590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:00.610797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-76e3d53f-a34a-4a89-8f04-81804be6f759 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:00.610797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-76e3d53f-a34a-4a89-8f04-81804be6f759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:00.622447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09f25e83fff24a8e924bb22e6f9fdd46', 'request_id': 'req-2f54a242-67a3-4106-a163-c2925a585991'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.642351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09f25e83fff24a8e924bb22e6f9fdd46', 'request_id': 'req-2f54a242-67a3-4106-a163-c2925a585991'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.653504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09f25e83fff24a8e924bb22e6f9fdd46', 'request_id': 'req-2f54a242-67a3-4106-a163-c2925a585991'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.666737 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] The token cache is being invalidate because project 09f25e83fff24a8e924bb22e6f9fdd46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:00.667053 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.667823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:00.669797 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1080/2155] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:00 2022] DELETE /identity/v3/projects/09f25e83fff24a8e924bb22e6f9fdd46 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:00.672529 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:00.677633 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:00.714998 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1076/2156] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:00.716728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab548fba-e52c-40fb-b711-12163803064e None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.717038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab548fba-e52c-40fb-b711-12163803064e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.717394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab548fba-e52c-40fb-b711-12163803064e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.759229 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ab548fba-e52c-40fb-b711-12163803064e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:00.762401 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ab548fba-e52c-40fb-b711-12163803064e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:00.774457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1077/2157] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:00.779439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab548fba-e52c-40fb-b711-12163803064e None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:00.793157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54a242-67a3-4106-a163-c2925a585991 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:00.793733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab548fba-e52c-40fb-b711-12163803064e None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:00.798064 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d0f1c-2221-42ae-891a-80fdb54e4d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:00.798678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d0f1c-2221-42ae-891a-80fdb54e4d94 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.798957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d0f1c-2221-42ae-891a-80fdb54e4d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.799102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3a12b-cbfb-4a29-9953-6f3f72c536b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:00.799300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d0f1c-2221-42ae-891a-80fdb54e4d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.800821 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3a12b-cbfb-4a29-9953-6f3f72c536b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.801122 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3a12b-cbfb-4a29-9953-6f3f72c536b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.801489 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3a12b-cbfb-4a29-9953-6f3f72c536b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.802151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3a12b-cbfb-4a29-9953-6f3f72c536b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:00.802465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3a12b-cbfb-4a29-9953-6f3f72c536b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:00.804177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3a12b-cbfb-4a29-9953-6f3f72c536b6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:00.804355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c60d0f1c-2221-42ae-891a-80fdb54e4d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:00.818604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1078/2158] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:50:00.819727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226059f6-d551-4739-937f-bc5ea62e6cd4 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.820014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226059f6-d551-4739-937f-bc5ea62e6cd4 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.820284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226059f6-d551-4739-937f-bc5ea62e6cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.820692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d0f1c-2221-42ae-891a-80fdb54e4d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:00.822802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d0f1c-2221-42ae-891a-80fdb54e4d94 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:00.842858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1081/2159] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:00.843718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-226059f6-d551-4739-937f-bc5ea62e6cd4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:00.844506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d0f1c-2221-42ae-891a-80fdb54e4d94 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:00.845113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-226059f6-d551-4739-937f-bc5ea62e6cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:00.858999 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1079/2160] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:00.860248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226059f6-d551-4739-937f-bc5ea62e6cd4 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:00.863018 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d0f1c-2221-42ae-891a-80fdb54e4d94 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:00.871639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226059f6-d551-4739-937f-bc5ea62e6cd4 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:00.874543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbae08-f27f-4c09-a936-2ee11f0fc4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:00.875706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbae08-f27f-4c09-a936-2ee11f0fc4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.876294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbae08-f27f-4c09-a936-2ee11f0fc4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.876796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbae08-f27f-4c09-a936-2ee11f0fc4b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.878506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b318479-6719-4fc4-ab7b-45898f800207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:00.879435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbae08-f27f-4c09-a936-2ee11f0fc4b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:00.879710 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b318479-6719-4fc4-ab7b-45898f800207 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.880144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b318479-6719-4fc4-ab7b-45898f800207 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.880590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b318479-6719-4fc4-ab7b-45898f800207 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.881413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b318479-6719-4fc4-ab7b-45898f800207 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:00.881866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b318479-6719-4fc4-ab7b-45898f800207 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:00.883583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbae08-f27f-4c09-a936-2ee11f0fc4b4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:00.883886 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b318479-6719-4fc4-ab7b-45898f800207 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:00.900032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1080/2161] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:50:00.905831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b318479-6719-4fc4-ab7b-45898f800207 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:00.919172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b318479-6719-4fc4-ab7b-45898f800207 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:00.926780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748c0d6-d2b5-4577-91ec-52051e8eacc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:00.927758 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748c0d6-d2b5-4577-91ec-52051e8eacc3 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:00.928038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748c0d6-d2b5-4577-91ec-52051e8eacc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:00.928307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748c0d6-d2b5-4577-91ec-52051e8eacc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:00.928990 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748c0d6-d2b5-4577-91ec-52051e8eacc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:00.931310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748c0d6-d2b5-4577-91ec-52051e8eacc3 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:00.941941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1082/2162] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:00 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:00.948397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbae08-f27f-4c09-a936-2ee11f0fc4b4 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:00.989540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1081/2163] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:00 2022] POST /identity/v3/projects => generated 389 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:00.993851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748c0d6-d2b5-4577-91ec-52051e8eacc3 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.037890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbae08-f27f-4c09-a936-2ee11f0fc4b4 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.091224 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748c0d6-d2b5-4577-91ec-52051e8eacc3 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.095747 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:01.097351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659c14-051c-4533-97ec-33260a5f58d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.098002 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659c14-051c-4533-97ec-33260a5f58d8 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.098324 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659c14-051c-4533-97ec-33260a5f58d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.098570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659c14-051c-4533-97ec-33260a5f58d8 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.099215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e659c14-051c-4533-97ec-33260a5f58d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.101281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e659c14-051c-4533-97ec-33260a5f58d8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.113775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d135407-d827-4823-ada7-6f8990783026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.114546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d135407-d827-4823-ada7-6f8990783026 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.114831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d135407-d827-4823-ada7-6f8990783026 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.115407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d135407-d827-4823-ada7-6f8990783026 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.115961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d135407-d827-4823-ada7-6f8990783026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.118102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d135407-d827-4823-ada7-6f8990783026 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.126663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1082/2164] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:00 2022] POST /identity/v3/users => generated 295 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:01.131337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659c14-051c-4533-97ec-33260a5f58d8 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.141519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659c14-051c-4533-97ec-33260a5f58d8 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.143850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1083/2165] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:00 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:01.147998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d135407-d827-4823-ada7-6f8990783026 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.151813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ecbb3-c8ff-4258-94c4-38e03c0451b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.152435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ecbb3-c8ff-4258-94c4-38e03c0451b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.152718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ecbb3-c8ff-4258-94c4-38e03c0451b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.153108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ecbb3-c8ff-4258-94c4-38e03c0451b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.153756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ecbb3-c8ff-4258-94c4-38e03c0451b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.155409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ecbb3-c8ff-4258-94c4-38e03c0451b0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.161298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d135407-d827-4823-ada7-6f8990783026 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.166222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fb4f5-ce4b-4140-9979-d738aeaa7d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.167168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fb4f5-ce4b-4140-9979-d738aeaa7d25 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.167168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fb4f5-ce4b-4140-9979-d738aeaa7d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.167381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fb4f5-ce4b-4140-9979-d738aeaa7d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.167520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1083/2166] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:50:01.171014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fb4f5-ce4b-4140-9979-d738aeaa7d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.171516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ecbb3-c8ff-4258-94c4-38e03c0451b0 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.172901 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fb4f5-ce4b-4140-9979-d738aeaa7d25 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.187836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ecbb3-c8ff-4258-94c4-38e03c0451b0 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.188540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1084/2167] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:50:01.193089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fb4f5-ce4b-4140-9979-d738aeaa7d25 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.194703 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61697066-a673-4a28-8611-68b2842e2572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.195384 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61697066-a673-4a28-8611-68b2842e2572 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.195663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61697066-a673-4a28-8611-68b2842e2572 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.195939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61697066-a673-4a28-8611-68b2842e2572 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.199525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61697066-a673-4a28-8611-68b2842e2572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.202210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61697066-a673-4a28-8611-68b2842e2572 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.208989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fb4f5-ce4b-4140-9979-d738aeaa7d25 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.219056 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce16b4-f684-4ec1-808a-2f235d3c26aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.220303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce16b4-f684-4ec1-808a-2f235d3c26aa admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.220790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce16b4-f684-4ec1-808a-2f235d3c26aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.221269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce16b4-f684-4ec1-808a-2f235d3c26aa admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.222993 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce16b4-f684-4ec1-808a-2f235d3c26aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.227382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1084/2168] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:50:01.227382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61697066-a673-4a28-8611-68b2842e2572 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.227616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce16b4-f684-4ec1-808a-2f235d3c26aa admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.229193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-61697066-a673-4a28-8611-68b2842e2572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:01.245921 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61697066-a673-4a28-8611-68b2842e2572 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.254148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1085/2169] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:50:01.254148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce16b4-f684-4ec1-808a-2f235d3c26aa admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.256738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19981c-f4c2-429f-bdba-434c2584b89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.257556 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19981c-f4c2-429f-bdba-434c2584b89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.257982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19981c-f4c2-429f-bdba-434c2584b89e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.258429 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19981c-f4c2-429f-bdba-434c2584b89e admin admin] PATH_INFO: `/v3/projects/97c88f12aeea4bed8ed0b3e3c58b7205` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.259773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19981c-f4c2-429f-bdba-434c2584b89e admin admin] RBAC: Authorizing `identity:delete_project(project_id=97c88f12aeea4bed8ed0b3e3c58b7205)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.261702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19981c-f4c2-429f-bdba-434c2584b89e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.263393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce16b4-f684-4ec1-808a-2f235d3c26aa admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.269700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121bb0c-6a9b-43cb-8cf2-d8ae28e9f191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.270311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121bb0c-6a9b-43cb-8cf2-d8ae28e9f191 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.270592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121bb0c-6a9b-43cb-8cf2-d8ae28e9f191 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.270864 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121bb0c-6a9b-43cb-8cf2-d8ae28e9f191 admin admin] PATH_INFO: `/v3/projects/40a8edca05bf4490948bb10b390bb373/users/18eecdfed8cc44f685095b6a70a4e69d/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.270967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:01.289491 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121bb0c-6a9b-43cb-8cf2-d8ae28e9f191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a8edca05bf4490948bb10b390bb373, user_id=18eecdfed8cc44f685095b6a70a4e69d, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.292347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121bb0c-6a9b-43cb-8cf2-d8ae28e9f191 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.314922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1086/2170] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:01 2022] PUT /identity/v3/projects/40a8edca05bf4490948bb10b390bb373/users/18eecdfed8cc44f685095b6a70a4e69d/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:01.314922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121bb0c-6a9b-43cb-8cf2-d8ae28e9f191 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.328866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0f19981c-f4c2-429f-bdba-434c2584b89e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97c88f12aeea4bed8ed0b3e3c58b7205', 'request_id': 'req-0f19981c-f4c2-429f-bdba-434c2584b89e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.336433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0f19981c-f4c2-429f-bdba-434c2584b89e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97c88f12aeea4bed8ed0b3e3c58b7205', 'request_id': 'req-0f19981c-f4c2-429f-bdba-434c2584b89e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.344153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0f19981c-f4c2-429f-bdba-434c2584b89e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97c88f12aeea4bed8ed0b3e3c58b7205', 'request_id': 'req-0f19981c-f4c2-429f-bdba-434c2584b89e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.370107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0f19981c-f4c2-429f-bdba-434c2584b89e admin admin] The token cache is being invalidate because project 97c88f12aeea4bed8ed0b3e3c58b7205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:01.370615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0f19981c-f4c2-429f-bdba-434c2584b89e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.371745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0f19981c-f4c2-429f-bdba-434c2584b89e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.374402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1085/2171] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:01 2022] DELETE /identity/v3/projects/97c88f12aeea4bed8ed0b3e3c58b7205 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:01.376458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e8495-569d-488a-b287-f41967b7f759 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.376745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e8495-569d-488a-b287-f41967b7f759 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.377759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e8495-569d-488a-b287-f41967b7f759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.384044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121bb0c-6a9b-43cb-8cf2-d8ae28e9f191 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.401239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ea0e8495-569d-488a-b287-f41967b7f759 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:01.402540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ea0e8495-569d-488a-b287-f41967b7f759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:01.449163 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a565d4b-f0b8-44b6-81d0-876322ea8a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.449842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a565d4b-f0b8-44b6-81d0-876322ea8a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.450123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a565d4b-f0b8-44b6-81d0-876322ea8a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.450402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a565d4b-f0b8-44b6-81d0-876322ea8a49 admin admin] PATH_INFO: `/v3/projects/07c33aad3a3a473bbaccc95c3cfc2517/users/7ee2fe869b8640229abf7f5b4c9278c4/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.469208 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a565d4b-f0b8-44b6-81d0-876322ea8a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07c33aad3a3a473bbaccc95c3cfc2517, user_id=7ee2fe869b8640229abf7f5b4c9278c4, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.472671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a565d4b-f0b8-44b6-81d0-876322ea8a49 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.480648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1086/2172] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:01.481701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e8495-569d-488a-b287-f41967b7f759 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.486271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ea0e8495-569d-488a-b287-f41967b7f759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:01.488669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1087/2173] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:01 2022] PUT /identity/v3/projects/07c33aad3a3a473bbaccc95c3cfc2517/users/7ee2fe869b8640229abf7f5b4c9278c4/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:01.494085 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a565d4b-f0b8-44b6-81d0-876322ea8a49 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.496083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7a565d4b-f0b8-44b6-81d0-876322ea8a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:01.511834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e8495-569d-488a-b287-f41967b7f759 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.536188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:01.572539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a565d4b-f0b8-44b6-81d0-876322ea8a49 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.578140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07d116-d97e-44cd-9c20-a5421af398b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.579001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07d116-d97e-44cd-9c20-a5421af398b4 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.579485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07d116-d97e-44cd-9c20-a5421af398b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.579943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07d116-d97e-44cd-9c20-a5421af398b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.586054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.586693 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.586966 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.587280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] PATH_INFO: `/v3/users/53ac7623b3b740179eb2022500ed2616` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.588742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53ac7623b3b740179eb2022500ed2616)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.590689 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.591931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07d116-d97e-44cd-9c20-a5421af398b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.593786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07d116-d97e-44cd-9c20-a5421af398b4 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.605648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1088/2174] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:01.606923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686c86a-698d-415b-a38f-0c14918c329e None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.607311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686c86a-698d-415b-a38f-0c14918c329e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.607589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686c86a-698d-415b-a38f-0c14918c329e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.638456 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e686c86a-698d-415b-a38f-0c14918c329e None None] MFA Rules not processed for user `7ee2fe869b8640229abf7f5b4c9278c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:01.639865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e686c86a-698d-415b-a38f-0c14918c329e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:01.640704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:01.654218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53ac7623b3b740179eb2022500ed2616'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.664148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53ac7623b3b740179eb2022500ed2616'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.665161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53ac7623b3b740179eb2022500ed2616'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.686748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53ac7623b3b740179eb2022500ed2616'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.687851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53ac7623b3b740179eb2022500ed2616'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.725400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1087/2175] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:01 2022] DELETE /identity/v3/users/53ac7623b3b740179eb2022500ed2616 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:01.746793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1089/2176] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:01.761614 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.763246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:01.889336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e932569-7f4e-4665-8ff3-0b4dd60feef6 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.900820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45be4f-8fd2-4d46-9d40-578cf5d21f4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.901826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45be4f-8fd2-4d46-9d40-578cf5d21f4c service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.902145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45be4f-8fd2-4d46-9d40-578cf5d21f4c service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.902449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45be4f-8fd2-4d46-9d40-578cf5d21f4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.979562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45be4f-8fd2-4d46-9d40-578cf5d21f4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.982254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45be4f-8fd2-4d46-9d40-578cf5d21f4c service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.994447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1088/2177] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:02.018496 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686c86a-698d-415b-a38f-0c14918c329e None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:02.021504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e686c86a-698d-415b-a38f-0c14918c329e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:02.045527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686c86a-698d-415b-a38f-0c14918c329e None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:02.123047 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:02.123964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:02.123964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:02.124157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] PATH_INFO: `/v3/users/7ee2fe869b8640229abf7f5b4c9278c4` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:02.125417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ee2fe869b8640229abf7f5b4c9278c4)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:02.127743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:02.216239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ee2fe869b8640229abf7f5b4c9278c4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.229455 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:02.244226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ee2fe869b8640229abf7f5b4c9278c4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.245065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ee2fe869b8640229abf7f5b4c9278c4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.245728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ee2fe869b8640229abf7f5b4c9278c4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.274023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ee2fe869b8640229abf7f5b4c9278c4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.318669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1090/2178] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:02 2022] DELETE /identity/v3/users/7ee2fe869b8640229abf7f5b4c9278c4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:02.325874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba7f82-2aaa-4c5e-aae4-25300aa1a3b4 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:02.326261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba7f82-2aaa-4c5e-aae4-25300aa1a3b4 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:02.326744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba7f82-2aaa-4c5e-aae4-25300aa1a3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:02.363551 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-81ba7f82-2aaa-4c5e-aae4-25300aa1a3b4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:02.365066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-81ba7f82-2aaa-4c5e-aae4-25300aa1a3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:02.483666 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1089/2179] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:02.496775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:02.498351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:02.523673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba7f82-2aaa-4c5e-aae4-25300aa1a3b4 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:02.526486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-81ba7f82-2aaa-4c5e-aae4-25300aa1a3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:02.548971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba7f82-2aaa-4c5e-aae4-25300aa1a3b4 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:02.582095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:02.583018 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:02.583972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:02.583972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] PATH_INFO: `/v3/projects/88a057a14c5d4411832317fb809dcd00` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:02.587808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88a057a14c5d4411832317fb809dcd00)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:02.591700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:02.596179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:02.604816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba079881-344a-4e73-9636-4d1f7b6745e6 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:02.609975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144ce54-571f-4068-a7a0-0aeecbe00389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:02.610597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144ce54-571f-4068-a7a0-0aeecbe00389 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:02.610875 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144ce54-571f-4068-a7a0-0aeecbe00389 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:02.611175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144ce54-571f-4068-a7a0-0aeecbe00389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:02.623168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144ce54-571f-4068-a7a0-0aeecbe00389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:02.624282 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144ce54-571f-4068-a7a0-0aeecbe00389 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:02.640113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1091/2180] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:02.684041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88a057a14c5d4411832317fb809dcd00', 'request_id': 'req-31829828-25c3-4b48-b578-192680d38207'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.701234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88a057a14c5d4411832317fb809dcd00', 'request_id': 'req-31829828-25c3-4b48-b578-192680d38207'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.715164 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88a057a14c5d4411832317fb809dcd00', 'request_id': 'req-31829828-25c3-4b48-b578-192680d38207'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.740715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] The token cache is being invalidate because project 88a057a14c5d4411832317fb809dcd00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:02.741146 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.742656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.756940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1090/2181] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:02 2022] DELETE /identity/v3/projects/88a057a14c5d4411832317fb809dcd00 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:02.762192 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144ce54-571f-4068-a7a0-0aeecbe00389 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:02.775847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144ce54-571f-4068-a7a0-0aeecbe00389 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:02.882942 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa110674-8675-4a5c-8f55-660ac1e64e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:02.883745 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa110674-8675-4a5c-8f55-660ac1e64e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:02.884642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa110674-8675-4a5c-8f55-660ac1e64e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:02.884991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa110674-8675-4a5c-8f55-660ac1e64e75 admin admin] PATH_INFO: `/v3/users/c7693d961dcb45c888089258ddb7c39b` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:02.886508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa110674-8675-4a5c-8f55-660ac1e64e75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7693d961dcb45c888089258ddb7c39b)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:02.888872 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa110674-8675-4a5c-8f55-660ac1e64e75 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:03.024795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-aa110674-8675-4a5c-8f55-660ac1e64e75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7693d961dcb45c888089258ddb7c39b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.044036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:03.045534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:03.074065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-aa110674-8675-4a5c-8f55-660ac1e64e75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7693d961dcb45c888089258ddb7c39b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.075037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-aa110674-8675-4a5c-8f55-660ac1e64e75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7693d961dcb45c888089258ddb7c39b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.075832 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-aa110674-8675-4a5c-8f55-660ac1e64e75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7693d961dcb45c888089258ddb7c39b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.082226 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31829828-25c3-4b48-b578-192680d38207 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:03.092225 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e483e-366d-49d1-a983-2db89acea038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:03.093283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e483e-366d-49d1-a983-2db89acea038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.095608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-aa110674-8675-4a5c-8f55-660ac1e64e75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7693d961dcb45c888089258ddb7c39b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.098053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e483e-366d-49d1-a983-2db89acea038 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.098346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e483e-366d-49d1-a983-2db89acea038 admin admin] PATH_INFO: `/v3/projects/07c33aad3a3a473bbaccc95c3cfc2517` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.102971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e483e-366d-49d1-a983-2db89acea038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07c33aad3a3a473bbaccc95c3cfc2517)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:03.105091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e483e-366d-49d1-a983-2db89acea038 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:03.112871 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:03.121150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1092/2182] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:02 2022] DELETE /identity/v3/users/c7693d961dcb45c888089258ddb7c39b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:03.226387 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-028e483e-366d-49d1-a983-2db89acea038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07c33aad3a3a473bbaccc95c3cfc2517', 'request_id': 'req-028e483e-366d-49d1-a983-2db89acea038'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.243626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-028e483e-366d-49d1-a983-2db89acea038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07c33aad3a3a473bbaccc95c3cfc2517', 'request_id': 'req-028e483e-366d-49d1-a983-2db89acea038'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.261201 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-028e483e-366d-49d1-a983-2db89acea038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07c33aad3a3a473bbaccc95c3cfc2517', 'request_id': 'req-028e483e-366d-49d1-a983-2db89acea038'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.284434 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-028e483e-366d-49d1-a983-2db89acea038 admin admin] The token cache is being invalidate because project 07c33aad3a3a473bbaccc95c3cfc2517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:03.285140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-028e483e-366d-49d1-a983-2db89acea038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.286130 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-028e483e-366d-49d1-a983-2db89acea038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.291731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1091/2183] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:03 2022] DELETE /identity/v3/projects/07c33aad3a3a473bbaccc95c3cfc2517 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:03.293581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d7a01-aeb4-4c1b-a5d7-341b239ce566 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.293890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d7a01-aeb4-4c1b-a5d7-341b239ce566 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.294220 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d7a01-aeb4-4c1b-a5d7-341b239ce566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.329514 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-544d7a01-aeb4-4c1b-a5d7-341b239ce566 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:03.333405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-544d7a01-aeb4-4c1b-a5d7-341b239ce566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:03.511500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1093/2184] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:03.519248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90272a61-e60b-44fa-827f-08f0976c2817 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.519936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90272a61-e60b-44fa-827f-08f0976c2817 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.519936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90272a61-e60b-44fa-827f-08f0976c2817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.566421 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-90272a61-e60b-44fa-827f-08f0976c2817 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:03.572570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-90272a61-e60b-44fa-827f-08f0976c2817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:03.596794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1092/2185] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:03.602897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d7a01-aeb4-4c1b-a5d7-341b239ce566 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:03.640927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d7a01-aeb4-4c1b-a5d7-341b239ce566 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:03.664079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90272a61-e60b-44fa-827f-08f0976c2817 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:03.671397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-90272a61-e60b-44fa-827f-08f0976c2817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:03.674974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119a1a9-043f-4ed7-93a4-c178e2eb9acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:03.675919 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119a1a9-043f-4ed7-93a4-c178e2eb9acd admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.676446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119a1a9-043f-4ed7-93a4-c178e2eb9acd admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.676999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119a1a9-043f-4ed7-93a4-c178e2eb9acd admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.678951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119a1a9-043f-4ed7-93a4-c178e2eb9acd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:03.679618 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119a1a9-043f-4ed7-93a4-c178e2eb9acd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:03.682530 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119a1a9-043f-4ed7-93a4-c178e2eb9acd admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:03.702962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90272a61-e60b-44fa-827f-08f0976c2817 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:03.723826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fca6a0-5977-465d-9e16-662a13dbd81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:03.726505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fca6a0-5977-465d-9e16-662a13dbd81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.727084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fca6a0-5977-465d-9e16-662a13dbd81e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.728557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fca6a0-5977-465d-9e16-662a13dbd81e admin admin] PATH_INFO: `/v3/projects/f26c60d5e2e140049abd076b71ba9f20` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.730625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fca6a0-5977-465d-9e16-662a13dbd81e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f26c60d5e2e140049abd076b71ba9f20)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:03.733804 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1094/2186] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:50:03.734970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fca6a0-5977-465d-9e16-662a13dbd81e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:03.740001 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119a1a9-043f-4ed7-93a4-c178e2eb9acd admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:03.761511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119a1a9-043f-4ed7-93a4-c178e2eb9acd admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:03.772681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ebd41-1008-4507-9bf5-8aafb89bf32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:03.773330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ebd41-1008-4507-9bf5-8aafb89bf32f admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.773606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ebd41-1008-4507-9bf5-8aafb89bf32f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.776379 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ebd41-1008-4507-9bf5-8aafb89bf32f admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.776731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ebd41-1008-4507-9bf5-8aafb89bf32f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:03.778688 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ebd41-1008-4507-9bf5-8aafb89bf32f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:03.833711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1095/2187] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:03 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:03.838908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ebd41-1008-4507-9bf5-8aafb89bf32f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:03.844355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-06fca6a0-5977-465d-9e16-662a13dbd81e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f26c60d5e2e140049abd076b71ba9f20', 'request_id': 'req-06fca6a0-5977-465d-9e16-662a13dbd81e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.859993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-06fca6a0-5977-465d-9e16-662a13dbd81e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f26c60d5e2e140049abd076b71ba9f20', 'request_id': 'req-06fca6a0-5977-465d-9e16-662a13dbd81e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.868253 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-06fca6a0-5977-465d-9e16-662a13dbd81e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f26c60d5e2e140049abd076b71ba9f20', 'request_id': 'req-06fca6a0-5977-465d-9e16-662a13dbd81e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.870277 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:03.896640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-06fca6a0-5977-465d-9e16-662a13dbd81e admin admin] The token cache is being invalidate because project f26c60d5e2e140049abd076b71ba9f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:03.898234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-06fca6a0-5977-465d-9e16-662a13dbd81e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.899248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-06fca6a0-5977-465d-9e16-662a13dbd81e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.901492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1093/2188] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:03 2022] DELETE /identity/v3/projects/f26c60d5e2e140049abd076b71ba9f20 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:03.913785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857e345-588d-4fb4-b6a2-62e677759d86 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.914322 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857e345-588d-4fb4-b6a2-62e677759d86 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.914742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857e345-588d-4fb4-b6a2-62e677759d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.947793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-e857e345-588d-4fb4-b6a2-62e677759d86 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:03.948465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ebd41-1008-4507-9bf5-8aafb89bf32f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:03.949277 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e857e345-588d-4fb4-b6a2-62e677759d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:03.957533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec25f4c-db02-4488-a5bb-a01d3eb29f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:03.958605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec25f4c-db02-4488-a5bb-a01d3eb29f7d admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.959207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec25f4c-db02-4488-a5bb-a01d3eb29f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.959637 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec25f4c-db02-4488-a5bb-a01d3eb29f7d admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.961492 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec25f4c-db02-4488-a5bb-a01d3eb29f7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:03.963662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1094/2189] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:03.965823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec25f4c-db02-4488-a5bb-a01d3eb29f7d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:03.982796 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564fe0c-01a9-4272-8967-203cee130b6c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.983362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564fe0c-01a9-4272-8967-203cee130b6c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.984234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564fe0c-01a9-4272-8967-203cee130b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.038909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a564fe0c-01a9-4272-8967-203cee130b6c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:04.042109 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a564fe0c-01a9-4272-8967-203cee130b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:04.044934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1096/2190] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:03 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:04.050452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec25f4c-db02-4488-a5bb-a01d3eb29f7d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:04.053081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-dec25f4c-db02-4488-a5bb-a01d3eb29f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:04.069967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1095/2191] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:04.073718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564fe0c-01a9-4272-8967-203cee130b6c None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:04.089176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564fe0c-01a9-4272-8967-203cee130b6c None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:04.093918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec25f4c-db02-4488-a5bb-a01d3eb29f7d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:04.096172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e082e-678e-4340-b428-61a2c0b4ee84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:04.097373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e082e-678e-4340-b428-61a2c0b4ee84 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.097763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e082e-678e-4340-b428-61a2c0b4ee84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.098058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e082e-678e-4340-b428-61a2c0b4ee84 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.098793 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e082e-678e-4340-b428-61a2c0b4ee84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:04.099104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e082e-678e-4340-b428-61a2c0b4ee84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:04.101059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e082e-678e-4340-b428-61a2c0b4ee84 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:04.109252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e90058-78a4-4ed7-bbca-fc8824735b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:04.111209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e90058-78a4-4ed7-bbca-fc8824735b04 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.114052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e90058-78a4-4ed7-bbca-fc8824735b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.114657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e90058-78a4-4ed7-bbca-fc8824735b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.115893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e90058-78a4-4ed7-bbca-fc8824735b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:04.118100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1096/2192] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:50:04.118755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e90058-78a4-4ed7-bbca-fc8824735b04 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:04.122302 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e082e-678e-4340-b428-61a2c0b4ee84 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:04.148818 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e082e-678e-4340-b428-61a2c0b4ee84 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:04.152295 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1097/2193] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:04.159353 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e90058-78a4-4ed7-bbca-fc8824735b04 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:04.179962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab835a40-32fc-42b0-8065-8909f70cad13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:04.180441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e90058-78a4-4ed7-bbca-fc8824735b04 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:04.180772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab835a40-32fc-42b0-8065-8909f70cad13 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.181125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab835a40-32fc-42b0-8065-8909f70cad13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.181473 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab835a40-32fc-42b0-8065-8909f70cad13 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.182339 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab835a40-32fc-42b0-8065-8909f70cad13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:04.185047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab835a40-32fc-42b0-8065-8909f70cad13 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:04.188566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e610b57-6af9-41e7-9828-846c37197256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:04.189754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e610b57-6af9-41e7-9828-846c37197256 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.190770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e610b57-6af9-41e7-9828-846c37197256 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.191479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e610b57-6af9-41e7-9828-846c37197256 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.192725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e610b57-6af9-41e7-9828-846c37197256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:04.195386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e610b57-6af9-41e7-9828-846c37197256 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:04.224689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1098/2194] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:04.263155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e610b57-6af9-41e7-9828-846c37197256 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:04.284435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1097/2195] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:04 2022] POST /identity/v3/projects => generated 363 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:04.284435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab835a40-32fc-42b0-8065-8909f70cad13 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:04.302478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e610b57-6af9-41e7-9828-846c37197256 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:04.362806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:04.403838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab835a40-32fc-42b0-8065-8909f70cad13 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:04.410163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6fdc9-bcbc-4bf5-ba56-111d4f273df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:04.410792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6fdc9-bcbc-4bf5-ba56-111d4f273df4 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.411075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6fdc9-bcbc-4bf5-ba56-111d4f273df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.411385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6fdc9-bcbc-4bf5-ba56-111d4f273df4 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.412041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6fdc9-bcbc-4bf5-ba56-111d4f273df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:04.414029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6fdc9-bcbc-4bf5-ba56-111d4f273df4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:04.417724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05805bd7-d15f-4bed-b06a-882e42f456c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:04.418910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05805bd7-d15f-4bed-b06a-882e42f456c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.419592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05805bd7-d15f-4bed-b06a-882e42f456c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.420127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05805bd7-d15f-4bed-b06a-882e42f456c9 admin admin] PATH_INFO: `/v3/projects/5e7da0316ba641e2a8829ba60d2ba561/users/b6ac1b4c398d45cc8bafb5bb821dfa8f/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.450106 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05805bd7-d15f-4bed-b06a-882e42f456c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e7da0316ba641e2a8829ba60d2ba561, user_id=b6ac1b4c398d45cc8bafb5bb821dfa8f, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:04.450615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1098/2196] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:04 2022] POST /identity/v3/users => generated 282 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:04.453450 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05805bd7-d15f-4bed-b06a-882e42f456c9 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:04.457090 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6fdc9-bcbc-4bf5-ba56-111d4f273df4 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:04.469497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6fdc9-bcbc-4bf5-ba56-111d4f273df4 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:04.478929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925446d-467d-44f5-95d6-9e10a0bad4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:04.479600 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925446d-467d-44f5-95d6-9e10a0bad4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.481399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925446d-467d-44f5-95d6-9e10a0bad4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.481711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925446d-467d-44f5-95d6-9e10a0bad4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.482459 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925446d-467d-44f5-95d6-9e10a0bad4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:04.484071 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925446d-467d-44f5-95d6-9e10a0bad4e9 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:04.487600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1099/2197] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:04 2022] PUT /identity/v3/projects/5e7da0316ba641e2a8829ba60d2ba561/users/b6ac1b4c398d45cc8bafb5bb821dfa8f/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:04.507508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d509bf-81ef-4828-9b92-f3bbd5d5ccfe None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.508178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d509bf-81ef-4828-9b92-f3bbd5d5ccfe None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.508836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d509bf-81ef-4828-9b92-f3bbd5d5ccfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.509774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1099/2198] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:04.513913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925446d-467d-44f5-95d6-9e10a0bad4e9 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:04.525901 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925446d-467d-44f5-95d6-9e10a0bad4e9 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:04.566997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-54d509bf-81ef-4828-9b92-f3bbd5d5ccfe None None] MFA Rules not processed for user `b6ac1b4c398d45cc8bafb5bb821dfa8f`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:04.569687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-54d509bf-81ef-4828-9b92-f3bbd5d5ccfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:04.614497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b73cf-fb75-4350-9362-cbaa1391e48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:04.615024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b73cf-fb75-4350-9362-cbaa1391e48a admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.615353 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b73cf-fb75-4350-9362-cbaa1391e48a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.615627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b73cf-fb75-4350-9362-cbaa1391e48a admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.616374 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b73cf-fb75-4350-9362-cbaa1391e48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:04.618576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b73cf-fb75-4350-9362-cbaa1391e48a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:04.637435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1100/2199] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:50:04.638899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b73cf-fb75-4350-9362-cbaa1391e48a admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:04.656162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b73cf-fb75-4350-9362-cbaa1391e48a admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:04.662584 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca155f4-17a1-4ba2-b50b-c74c8763c655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:04.663468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca155f4-17a1-4ba2-b50b-c74c8763c655 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.663815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca155f4-17a1-4ba2-b50b-c74c8763c655 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.664164 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca155f4-17a1-4ba2-b50b-c74c8763c655 admin admin] PATH_INFO: `/v3/projects/ff4823e4fae64d1cbae8bfef6ac062fc/users/02b26d233b7b440da825dfc619c90e8f/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.682418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca155f4-17a1-4ba2-b50b-c74c8763c655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4823e4fae64d1cbae8bfef6ac062fc, user_id=02b26d233b7b440da825dfc619c90e8f, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:04.691563 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca155f4-17a1-4ba2-b50b-c74c8763c655 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:04.694206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:04.708818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1100/2200] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:04.720447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1101/2201] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:04 2022] PUT /identity/v3/projects/ff4823e4fae64d1cbae8bfef6ac062fc/users/02b26d233b7b440da825dfc619c90e8f/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:04.723237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d509bf-81ef-4828-9b92-f3bbd5d5ccfe None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:04.726613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-54d509bf-81ef-4828-9b92-f3bbd5d5ccfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:04.743671 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579df245-c7b3-4d8d-af79-4bb32c393ca9 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.744423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579df245-c7b3-4d8d-af79-4bb32c393ca9 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.745550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579df245-c7b3-4d8d-af79-4bb32c393ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.818549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-579df245-c7b3-4d8d-af79-4bb32c393ca9 None None] MFA Rules not processed for user `02b26d233b7b440da825dfc619c90e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:04.822048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-579df245-c7b3-4d8d-af79-4bb32c393ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:04.938513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d509bf-81ef-4828-9b92-f3bbd5d5ccfe None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:04.985049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1102/2202] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:05.001192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579df245-c7b3-4d8d-af79-4bb32c393ca9 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:05.007311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-579df245-c7b3-4d8d-af79-4bb32c393ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:05.114275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a87a2-d89f-4ad7-85ed-4e0ccea8de09 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06fad9466df14827983dac2b4b844f3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:05.114926 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a87a2-d89f-4ad7-85ed-4e0ccea8de09 service glance] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:05.115241 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a87a2-d89f-4ad7-85ed-4e0ccea8de09 service glance] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:05.115524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a87a2-d89f-4ad7-85ed-4e0ccea8de09 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:05.199098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:05.242578 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579df245-c7b3-4d8d-af79-4bb32c393ca9 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:05.271342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1c9b4-b8c0-4817-b2d6-f53379a1c5ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:05.271970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1c9b4-b8c0-4817-b2d6-f53379a1c5ee service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:05.272243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1c9b4-b8c0-4817-b2d6-f53379a1c5ee service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:05.272505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1c9b4-b8c0-4817-b2d6-f53379a1c5ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:05.308025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1c9b4-b8c0-4817-b2d6-f53379a1c5ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:05.309500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1c9b4-b8c0-4817-b2d6-f53379a1c5ee service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:05.337486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1103/2203] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:05.386597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a87a2-d89f-4ad7-85ed-4e0ccea8de09 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:05.388218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a87a2-d89f-4ad7-85ed-4e0ccea8de09 service glance] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:05.411047 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1101/2204] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:08.389281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faba708-ab82-48e1-ab46-5c0dc521157c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:08.390098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faba708-ab82-48e1-ab46-5c0dc521157c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:08.390098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faba708-ab82-48e1-ab46-5c0dc521157c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:08.463175 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4faba708-ab82-48e1-ab46-5c0dc521157c None None] MFA Rules not processed for user `18eecdfed8cc44f685095b6a70a4e69d`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:08.464366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4faba708-ab82-48e1-ab46-5c0dc521157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:08.552207 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1104/2205] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:08.565986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a87a2-d89f-4ad7-85ed-4e0ccea8de09 service glance] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:08.567716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-8c3a87a2-d89f-4ad7-85ed-4e0ccea8de09 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:08.661490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a87a2-d89f-4ad7-85ed-4e0ccea8de09 service glance] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:08.672386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999b2f5-c49e-46f9-b890-49ec79eef8f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:08.672950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999b2f5-c49e-46f9-b890-49ec79eef8f0 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:08.673424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999b2f5-c49e-46f9-b890-49ec79eef8f0 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:08.673899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999b2f5-c49e-46f9-b890-49ec79eef8f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:08.689045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999b2f5-c49e-46f9-b890-49ec79eef8f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:08.692846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999b2f5-c49e-46f9-b890-49ec79eef8f0 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:08.706173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1102/2206] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:10.505705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead37269-9c5e-49f8-becc-d12a4554e9d5 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:10.506316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead37269-9c5e-49f8-becc-d12a4554e9d5 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:10.507085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead37269-9c5e-49f8-becc-d12a4554e9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:10.550142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ead37269-9c5e-49f8-becc-d12a4554e9d5 None None] MFA Rules not processed for user `18eecdfed8cc44f685095b6a70a4e69d`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:10.552676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ead37269-9c5e-49f8-becc-d12a4554e9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:10.566747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1105/2207] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:15.806261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999b2f5-c49e-46f9-b890-49ec79eef8f0 service nova] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:15.807782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3999b2f5-c49e-46f9-b890-49ec79eef8f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:15.893255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999b2f5-c49e-46f9-b890-49ec79eef8f0 service nova] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:15.898325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:15.899025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:15.899362 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:15.899635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] PATH_INFO: `/v3/users/7f90e8a8e998441f8955513cbb8fe4b0` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:15.900838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f90e8a8e998441f8955513cbb8fe4b0)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:15.902781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:15.970897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:15.980240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f90e8a8e998441f8955513cbb8fe4b0'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:15.996098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f90e8a8e998441f8955513cbb8fe4b0'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:15.996850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f90e8a8e998441f8955513cbb8fe4b0'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:15.997639 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f90e8a8e998441f8955513cbb8fe4b0'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:16.007624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f90e8a8e998441f8955513cbb8fe4b0'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:16.025466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1103/2208] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:15 2022] DELETE /identity/v3/users/7f90e8a8e998441f8955513cbb8fe4b0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:16.030440 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2981f-0d75-47d4-94ac-5273ae3e662a None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:16.030929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2981f-0d75-47d4-94ac-5273ae3e662a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:16.031390 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2981f-0d75-47d4-94ac-5273ae3e662a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:16.060574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-64b2981f-0d75-47d4-94ac-5273ae3e662a None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:16.062140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-64b2981f-0d75-47d4-94ac-5273ae3e662a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:16.073439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:16.137616 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1106/2209] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:16.148218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:16.150400 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:16.227113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c5782-43a3-4191-8943-6738d7d0ea37 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:16.231396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb4cd2-bf5b-4192-9ebd-e3886385da29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:16.232363 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb4cd2-bf5b-4192-9ebd-e3886385da29 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:16.232648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb4cd2-bf5b-4192-9ebd-e3886385da29 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:16.232994 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb4cd2-bf5b-4192-9ebd-e3886385da29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:16.245393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb4cd2-bf5b-4192-9ebd-e3886385da29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:16.246914 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb4cd2-bf5b-4192-9ebd-e3886385da29 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:16.258121 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1104/2210] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:16.742951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2981f-0d75-47d4-94ac-5273ae3e662a None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:16.745494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-64b2981f-0d75-47d4-94ac-5273ae3e662a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:16.790000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2981f-0d75-47d4-94ac-5273ae3e662a None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:16.798602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:16.800297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:16.800858 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:16.801443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27 admin admin] PATH_INFO: `/v3/projects/11b846012de64206b9fd987b6c41ecc7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:16.803733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11b846012de64206b9fd987b6c41ecc7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:16.807603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:16.902627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11b846012de64206b9fd987b6c41ecc7', 'request_id': 'req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:16.913953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11b846012de64206b9fd987b6c41ecc7', 'request_id': 'req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:16.925059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11b846012de64206b9fd987b6c41ecc7', 'request_id': 'req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:16.944692 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27 admin admin] The token cache is being invalidate because project 11b846012de64206b9fd987b6c41ecc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:16.945137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:16.945925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ab8bc42c-68ec-43dc-bb64-b4dd74798d27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:16.947810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1107/2211] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:16 2022] DELETE /identity/v3/projects/11b846012de64206b9fd987b6c41ecc7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:16.955968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c5740-554f-4489-a31c-bb98ade6e380 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:16.956638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c5740-554f-4489-a31c-bb98ade6e380 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:16.956638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c5740-554f-4489-a31c-bb98ade6e380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:16.980113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-215c5740-554f-4489-a31c-bb98ade6e380 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:16.981364 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-215c5740-554f-4489-a31c-bb98ade6e380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:17.050878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1105/2212] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:17.059146 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe25d1-fe97-4128-bc88-463d5e040699 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:17.059715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe25d1-fe97-4128-bc88-463d5e040699 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:17.059817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe25d1-fe97-4128-bc88-463d5e040699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:17.082902 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-e9fe25d1-fe97-4128-bc88-463d5e040699 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:17.084289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e9fe25d1-fe97-4128-bc88-463d5e040699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:17.093233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1108/2213] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:17.098365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c5740-554f-4489-a31c-bb98ade6e380 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:17.111202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c5740-554f-4489-a31c-bb98ade6e380 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:17.120665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701561c-e815-4834-aa9d-7f60c6321cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:17.121577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701561c-e815-4834-aa9d-7f60c6321cee admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:17.121946 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701561c-e815-4834-aa9d-7f60c6321cee admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:17.122620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701561c-e815-4834-aa9d-7f60c6321cee admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:17.123844 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701561c-e815-4834-aa9d-7f60c6321cee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:17.124352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701561c-e815-4834-aa9d-7f60c6321cee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:17.126802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701561c-e815-4834-aa9d-7f60c6321cee admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:17.149365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1106/2214] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:50:17.154833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe25d1-fe97-4128-bc88-463d5e040699 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:17.170962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe25d1-fe97-4128-bc88-463d5e040699 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:17.187259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6293e4c-58cc-4a03-98f9-19e1f50076e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:17.188289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6293e4c-58cc-4a03-98f9-19e1f50076e0 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:17.188669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6293e4c-58cc-4a03-98f9-19e1f50076e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:17.188947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6293e4c-58cc-4a03-98f9-19e1f50076e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:17.189972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6293e4c-58cc-4a03-98f9-19e1f50076e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:17.193958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6293e4c-58cc-4a03-98f9-19e1f50076e0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:17.237731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1109/2215] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:17 2022] POST /identity/v3/projects => generated 375 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:17.242652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701561c-e815-4834-aa9d-7f60c6321cee admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:17.257130 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701561c-e815-4834-aa9d-7f60c6321cee admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:17.327481 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:17.343655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cfa44-663b-420d-a13e-b0ac2d5920f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:17.344487 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cfa44-663b-420d-a13e-b0ac2d5920f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:17.344891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cfa44-663b-420d-a13e-b0ac2d5920f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:17.345256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cfa44-663b-420d-a13e-b0ac2d5920f2 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:17.346088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cfa44-663b-420d-a13e-b0ac2d5920f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:17.349242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cfa44-663b-420d-a13e-b0ac2d5920f2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:17.376772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1107/2216] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:17 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:17.382095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6293e4c-58cc-4a03-98f9-19e1f50076e0 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:17.393450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6293e4c-58cc-4a03-98f9-19e1f50076e0 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:17.400214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc6e25-653a-4e08-b8d7-bf2c58977278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:17.400839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc6e25-653a-4e08-b8d7-bf2c58977278 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:17.401113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc6e25-653a-4e08-b8d7-bf2c58977278 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:17.401390 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc6e25-653a-4e08-b8d7-bf2c58977278 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:17.402038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc6e25-653a-4e08-b8d7-bf2c58977278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:17.403734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc6e25-653a-4e08-b8d7-bf2c58977278 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:17.415084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1110/2217] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:17.419594 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cfa44-663b-420d-a13e-b0ac2d5920f2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:17.432300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cfa44-663b-420d-a13e-b0ac2d5920f2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:17.437240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cbe53-9505-4ae8-b3ee-2c587b9965e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:17.437858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cbe53-9505-4ae8-b3ee-2c587b9965e5 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:17.438134 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cbe53-9505-4ae8-b3ee-2c587b9965e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:17.438572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cbe53-9505-4ae8-b3ee-2c587b9965e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:17.439259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cbe53-9505-4ae8-b3ee-2c587b9965e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:17.442105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cbe53-9505-4ae8-b3ee-2c587b9965e5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:17.454282 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1108/2218] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:17.459778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc6e25-653a-4e08-b8d7-bf2c58977278 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:17.472472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc6e25-653a-4e08-b8d7-bf2c58977278 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:17.478037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3234b7-7889-489e-a2ce-40320c536e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:17.478720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3234b7-7889-489e-a2ce-40320c536e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:17.479003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3234b7-7889-489e-a2ce-40320c536e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:17.479318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3234b7-7889-489e-a2ce-40320c536e83 admin admin] PATH_INFO: `/v3/projects/c88af70f29db4015a6fce06089debe5c/users/646da13f9db94b0fa2302d6b45774e54/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:17.516842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3234b7-7889-489e-a2ce-40320c536e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c88af70f29db4015a6fce06089debe5c, user_id=646da13f9db94b0fa2302d6b45774e54, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:17.520728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3234b7-7889-489e-a2ce-40320c536e83 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:17.538778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1111/2219] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:17 2022] PUT /identity/v3/projects/c88af70f29db4015a6fce06089debe5c/users/646da13f9db94b0fa2302d6b45774e54/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:17.559990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7262ce-b3f2-4511-8702-7d3a4f9cb0a9 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:17.560508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7262ce-b3f2-4511-8702-7d3a4f9cb0a9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:17.561121 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7262ce-b3f2-4511-8702-7d3a4f9cb0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:17.605515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-fc7262ce-b3f2-4511-8702-7d3a4f9cb0a9 None None] MFA Rules not processed for user `646da13f9db94b0fa2302d6b45774e54`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:17.607693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-fc7262ce-b3f2-4511-8702-7d3a4f9cb0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:17.734312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1109/2220] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:17.750279 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3234b7-7889-489e-a2ce-40320c536e83 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:17.752859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9a3234b7-7889-489e-a2ce-40320c536e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:17.789801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:17.827712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3234b7-7889-489e-a2ce-40320c536e83 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:17.832212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366876e6-4d72-40b0-a8df-5d903560cbce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:17.832836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366876e6-4d72-40b0-a8df-5d903560cbce service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:17.833106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366876e6-4d72-40b0-a8df-5d903560cbce service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:17.833375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366876e6-4d72-40b0-a8df-5d903560cbce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:17.843114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366876e6-4d72-40b0-a8df-5d903560cbce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:17.844787 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366876e6-4d72-40b0-a8df-5d903560cbce service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:17.859173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1112/2221] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:20.378496 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70cd59-c402-4839-b22e-83b63476aaac None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:20.379017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70cd59-c402-4839-b22e-83b63476aaac None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:20.379017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70cd59-c402-4839-b22e-83b63476aaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:20.401010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ae70cd59-c402-4839-b22e-83b63476aaac None None] MFA Rules not processed for user `646da13f9db94b0fa2302d6b45774e54`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:20.402278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ae70cd59-c402-4839-b22e-83b63476aaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:20.410435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1110/2222] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:27.851350 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=13 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:30.398318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:36.941333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366876e6-4d72-40b0-a8df-5d903560cbce service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:36.943664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-366876e6-4d72-40b0-a8df-5d903560cbce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:37.039297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366876e6-4d72-40b0-a8df-5d903560cbce service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:37.043959 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:37.044604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:37.044881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:37.045156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] PATH_INFO: `/v3/users/8d19f42081f3422c947cfbabc40d94ea` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:37.046353 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d19f42081f3422c947cfbabc40d94ea)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:37.048435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:37.110378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d19f42081f3422c947cfbabc40d94ea'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:37.121236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d19f42081f3422c947cfbabc40d94ea'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:37.121941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d19f42081f3422c947cfbabc40d94ea'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:37.140519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d19f42081f3422c947cfbabc40d94ea'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:37.141397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d19f42081f3422c947cfbabc40d94ea'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:37.165172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1113/2223] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:36 2022] DELETE /identity/v3/users/8d19f42081f3422c947cfbabc40d94ea => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:37.172203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a7036-53a7-4886-9e62-d37d3758a08b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:37.172717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a7036-53a7-4886-9e62-d37d3758a08b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:37.172717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a7036-53a7-4886-9e62-d37d3758a08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:37.205496 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b67a7036-53a7-4886-9e62-d37d3758a08b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:37.208657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b67a7036-53a7-4886-9e62-d37d3758a08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:37.307254 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1111/2224] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:37.320303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:37.322618 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:37.419264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caa5f7-dfc7-4138-8bd8-5298481a4c6b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:37.425999 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f07ac-89e4-4245-9122-dc14ead88552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:37.427040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f07ac-89e4-4245-9122-dc14ead88552 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:37.427688 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f07ac-89e4-4245-9122-dc14ead88552 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:37.428182 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f07ac-89e4-4245-9122-dc14ead88552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:37.445512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f07ac-89e4-4245-9122-dc14ead88552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:37.447026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f07ac-89e4-4245-9122-dc14ead88552 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:37.460901 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1114/2225] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:38.330906 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a7036-53a7-4886-9e62-d37d3758a08b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:38.334005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b67a7036-53a7-4886-9e62-d37d3758a08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:38.363206 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a7036-53a7-4886-9e62-d37d3758a08b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:38.375413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:38.376746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:38.377477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:38.377969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391 admin admin] PATH_INFO: `/v3/projects/1cdc3c67b3ae461ab319c5e3530dc4bc` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:38.380595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cdc3c67b3ae461ab319c5e3530dc4bc)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:38.385525 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:38.473419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cdc3c67b3ae461ab319c5e3530dc4bc', 'request_id': 'req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:38.487428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cdc3c67b3ae461ab319c5e3530dc4bc', 'request_id': 'req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:38.502798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cdc3c67b3ae461ab319c5e3530dc4bc', 'request_id': 'req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:38.524781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391 admin admin] The token cache is being invalidate because project 1cdc3c67b3ae461ab319c5e3530dc4bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:38.531771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:38.533309 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ebbb21ed-d92b-4836-ba8a-ce3d9d3b7391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:38.544171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1112/2226] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:38 2022] DELETE /identity/v3/projects/1cdc3c67b3ae461ab319c5e3530dc4bc => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:38.545576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1b68a-b867-4f35-b11d-690e63aa8fbf None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:38.546314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1b68a-b867-4f35-b11d-690e63aa8fbf None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:38.546314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1b68a-b867-4f35-b11d-690e63aa8fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:38.578821 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-05e1b68a-b867-4f35-b11d-690e63aa8fbf None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:38.581011 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-05e1b68a-b867-4f35-b11d-690e63aa8fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:38.618404 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:38.709864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1115/2227] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:38.724358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dac146-0433-4724-8ecb-c86cef0d7b34 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:38.724779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dac146-0433-4724-8ecb-c86cef0d7b34 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:38.725285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dac146-0433-4724-8ecb-c86cef0d7b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:38.758285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-33dac146-0433-4724-8ecb-c86cef0d7b34 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:38.759499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-33dac146-0433-4724-8ecb-c86cef0d7b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:38.768267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1113/2228] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:38.773728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1b68a-b867-4f35-b11d-690e63aa8fbf None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:38.784560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1b68a-b867-4f35-b11d-690e63aa8fbf None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:38.791060 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8f3cd-f08e-4634-93de-37ec59683dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:38.791848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8f3cd-f08e-4634-93de-37ec59683dae admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:38.792240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8f3cd-f08e-4634-93de-37ec59683dae admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:38.792571 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8f3cd-f08e-4634-93de-37ec59683dae admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:38.793296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8f3cd-f08e-4634-93de-37ec59683dae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:38.794320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8f3cd-f08e-4634-93de-37ec59683dae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:38.796104 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8f3cd-f08e-4634-93de-37ec59683dae admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:38.815373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1116/2229] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:50:38.820749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dac146-0433-4724-8ecb-c86cef0d7b34 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:38.831048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dac146-0433-4724-8ecb-c86cef0d7b34 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:38.836562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca76f3f-58bb-4983-8d60-d1cb273a9254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:38.837194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca76f3f-58bb-4983-8d60-d1cb273a9254 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:38.837473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca76f3f-58bb-4983-8d60-d1cb273a9254 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:38.837741 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca76f3f-58bb-4983-8d60-d1cb273a9254 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:38.838462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca76f3f-58bb-4983-8d60-d1cb273a9254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:38.840540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca76f3f-58bb-4983-8d60-d1cb273a9254 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:38.887548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1114/2230] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:38 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:38.888817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8f3cd-f08e-4634-93de-37ec59683dae admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:38.901193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8f3cd-f08e-4634-93de-37ec59683dae admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:38.995812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4910445-2edd-476b-8688-1ed03f86ae09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:38.999553 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4910445-2edd-476b-8688-1ed03f86ae09 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:39.000243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4910445-2edd-476b-8688-1ed03f86ae09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:39.000827 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4910445-2edd-476b-8688-1ed03f86ae09 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:39.002645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4910445-2edd-476b-8688-1ed03f86ae09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:39.006499 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4910445-2edd-476b-8688-1ed03f86ae09 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:39.060899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca76f3f-58bb-4983-8d60-d1cb273a9254 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:39.076533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1117/2231] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:38 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:39.105266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca76f3f-58bb-4983-8d60-d1cb273a9254 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:39.114005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86fa1d-b95b-4f1e-b60a-bff15ab22924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:39.115383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86fa1d-b95b-4f1e-b60a-bff15ab22924 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:39.115890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86fa1d-b95b-4f1e-b60a-bff15ab22924 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:39.116308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86fa1d-b95b-4f1e-b60a-bff15ab22924 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:39.117105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86fa1d-b95b-4f1e-b60a-bff15ab22924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:39.119581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86fa1d-b95b-4f1e-b60a-bff15ab22924 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:39.135824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1115/2232] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:50:39.136920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4910445-2edd-476b-8688-1ed03f86ae09 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:39.148194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4910445-2edd-476b-8688-1ed03f86ae09 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:39.157276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a3a4e-d75a-41f8-9393-4bad94ecfb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:39.157276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a3a4e-d75a-41f8-9393-4bad94ecfb98 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:39.157276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a3a4e-d75a-41f8-9393-4bad94ecfb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:39.157276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a3a4e-d75a-41f8-9393-4bad94ecfb98 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:39.157848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a3a4e-d75a-41f8-9393-4bad94ecfb98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:39.157848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a3a4e-d75a-41f8-9393-4bad94ecfb98 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:39.171946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1118/2233] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:50:39.176283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86fa1d-b95b-4f1e-b60a-bff15ab22924 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:39.187025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86fa1d-b95b-4f1e-b60a-bff15ab22924 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:39.193077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe443b-a31f-49cf-9204-d3db08cdc05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:39.193855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe443b-a31f-49cf-9204-d3db08cdc05d admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:39.194186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe443b-a31f-49cf-9204-d3db08cdc05d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:39.194527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe443b-a31f-49cf-9204-d3db08cdc05d admin admin] PATH_INFO: `/v3/projects/7d45013dccd34a8994a4b25c6ac0f2bf/users/bf45320c49bd443388f5af0e3871bc2e/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:39.211240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe443b-a31f-49cf-9204-d3db08cdc05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d45013dccd34a8994a4b25c6ac0f2bf, user_id=bf45320c49bd443388f5af0e3871bc2e, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:39.214072 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe443b-a31f-49cf-9204-d3db08cdc05d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:39.231608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1116/2234] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:39 2022] PUT /identity/v3/projects/7d45013dccd34a8994a4b25c6ac0f2bf/users/bf45320c49bd443388f5af0e3871bc2e/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:39.244836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46877f-adf6-4d3f-8057-4296a61b8d09 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:39.245129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46877f-adf6-4d3f-8057-4296a61b8d09 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:39.245519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46877f-adf6-4d3f-8057-4296a61b8d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:39.285705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-6e46877f-adf6-4d3f-8057-4296a61b8d09 None None] MFA Rules not processed for user `bf45320c49bd443388f5af0e3871bc2e`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:39.287242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6e46877f-adf6-4d3f-8057-4296a61b8d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:39.415364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1119/2235] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:39.436088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe443b-a31f-49cf-9204-d3db08cdc05d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:39.439150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-14fe443b-a31f-49cf-9204-d3db08cdc05d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:39.481537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:39.528415 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe443b-a31f-49cf-9204-d3db08cdc05d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:39.532331 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd325285-d753-4292-8844-85438f9d4ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:39.532965 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd325285-d753-4292-8844-85438f9d4ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:39.533263 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd325285-d753-4292-8844-85438f9d4ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:39.533521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd325285-d753-4292-8844-85438f9d4ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:39.557408 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd325285-d753-4292-8844-85438f9d4ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:39.559068 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd325285-d753-4292-8844-85438f9d4ad2 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:39.570409 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1117/2236] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:41.456012 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46877f-adf6-4d3f-8057-4296a61b8d09 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:41.458889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6e46877f-adf6-4d3f-8057-4296a61b8d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:41.497229 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:41.582102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46877f-adf6-4d3f-8057-4296a61b8d09 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:41.589278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:41.590847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:41.591375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:41.591854 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] PATH_INFO: `/v3/users/b6ac1b4c398d45cc8bafb5bb821dfa8f` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:41.593608 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ac1b4c398d45cc8bafb5bb821dfa8f)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:41.597107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:41.674129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ac1b4c398d45cc8bafb5bb821dfa8f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:41.685232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ac1b4c398d45cc8bafb5bb821dfa8f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:41.685909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ac1b4c398d45cc8bafb5bb821dfa8f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:41.706089 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ac1b4c398d45cc8bafb5bb821dfa8f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:41.707504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ac1b4c398d45cc8bafb5bb821dfa8f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:41.731359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1120/2237] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:41 2022] DELETE /identity/v3/users/b6ac1b4c398d45cc8bafb5bb821dfa8f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:41.731464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2808c2-a56f-4116-9b98-94cef242f34c None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:41.731464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2808c2-a56f-4116-9b98-94cef242f34c None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:41.731464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2808c2-a56f-4116-9b98-94cef242f34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:41.764432 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-af2808c2-a56f-4116-9b98-94cef242f34c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:41.765698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-af2808c2-a56f-4116-9b98-94cef242f34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:41.871908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1118/2238] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:41.884014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:41.885792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:41.983637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b74c5-51ff-4b86-abfd-15a2a0b42d6d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:41.992486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7f681-f257-42d9-ad98-28e16e2425f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:41.994354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7f681-f257-42d9-ad98-28e16e2425f7 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:41.994924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7f681-f257-42d9-ad98-28e16e2425f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:41.995615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7f681-f257-42d9-ad98-28e16e2425f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:42.009424 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7f681-f257-42d9-ad98-28e16e2425f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:42.011074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7f681-f257-42d9-ad98-28e16e2425f7 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:42.028211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1121/2239] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:42.355200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2808c2-a56f-4116-9b98-94cef242f34c None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:42.357019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-af2808c2-a56f-4116-9b98-94cef242f34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:42.387619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2808c2-a56f-4116-9b98-94cef242f34c None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:42.395234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688de2df-6825-4414-9bda-df99e788d642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:42.396220 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688de2df-6825-4414-9bda-df99e788d642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:42.396773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688de2df-6825-4414-9bda-df99e788d642 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:42.397237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688de2df-6825-4414-9bda-df99e788d642 admin admin] PATH_INFO: `/v3/projects/5e7da0316ba641e2a8829ba60d2ba561` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:42.399075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688de2df-6825-4414-9bda-df99e788d642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e7da0316ba641e2a8829ba60d2ba561)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:42.402087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688de2df-6825-4414-9bda-df99e788d642 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:42.475383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-688de2df-6825-4414-9bda-df99e788d642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e7da0316ba641e2a8829ba60d2ba561', 'request_id': 'req-688de2df-6825-4414-9bda-df99e788d642'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:42.486358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-688de2df-6825-4414-9bda-df99e788d642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e7da0316ba641e2a8829ba60d2ba561', 'request_id': 'req-688de2df-6825-4414-9bda-df99e788d642'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:42.496147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-688de2df-6825-4414-9bda-df99e788d642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e7da0316ba641e2a8829ba60d2ba561', 'request_id': 'req-688de2df-6825-4414-9bda-df99e788d642'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:42.516221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-688de2df-6825-4414-9bda-df99e788d642 admin admin] The token cache is being invalidate because project 5e7da0316ba641e2a8829ba60d2ba561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:42.516697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-688de2df-6825-4414-9bda-df99e788d642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:42.517812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-688de2df-6825-4414-9bda-df99e788d642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:42.525020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1119/2240] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:42 2022] DELETE /identity/v3/projects/5e7da0316ba641e2a8829ba60d2ba561 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:42.527347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebbe92-09ee-42e0-bede-7b3ae603d869 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:42.527892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebbe92-09ee-42e0-bede-7b3ae603d869 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:42.527892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebbe92-09ee-42e0-bede-7b3ae603d869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:42.555517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a7ebbe92-09ee-42e0-bede-7b3ae603d869 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:42.556912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a7ebbe92-09ee-42e0-bede-7b3ae603d869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:42.635209 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:42.668769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1122/2241] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:42.680371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16496cf9-177b-433a-a97d-1803df699b78 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:42.680751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16496cf9-177b-433a-a97d-1803df699b78 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:42.681161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16496cf9-177b-433a-a97d-1803df699b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:42.703807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-16496cf9-177b-433a-a97d-1803df699b78 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:42.705100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-16496cf9-177b-433a-a97d-1803df699b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:42.716761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1120/2242] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:42.719935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebbe92-09ee-42e0-bede-7b3ae603d869 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:42.733220 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebbe92-09ee-42e0-bede-7b3ae603d869 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:42.740515 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38558fa9-5c99-4764-9cfb-5331013fc27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:42.741169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38558fa9-5c99-4764-9cfb-5331013fc27d admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:42.741450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38558fa9-5c99-4764-9cfb-5331013fc27d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:42.741717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38558fa9-5c99-4764-9cfb-5331013fc27d admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:42.742484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38558fa9-5c99-4764-9cfb-5331013fc27d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:42.742802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38558fa9-5c99-4764-9cfb-5331013fc27d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:42.744698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38558fa9-5c99-4764-9cfb-5331013fc27d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:42.759904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1123/2243] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:50:42.764696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16496cf9-177b-433a-a97d-1803df699b78 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:42.776364 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16496cf9-177b-433a-a97d-1803df699b78 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:42.781935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738171d-e52a-4918-9077-193d178ca0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:42.782519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738171d-e52a-4918-9077-193d178ca0ff admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:42.782800 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738171d-e52a-4918-9077-193d178ca0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:42.783096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738171d-e52a-4918-9077-193d178ca0ff admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:42.783812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738171d-e52a-4918-9077-193d178ca0ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:42.786625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738171d-e52a-4918-9077-193d178ca0ff admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:42.826403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1121/2244] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:42 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:42.833310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38558fa9-5c99-4764-9cfb-5331013fc27d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:42.844457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38558fa9-5c99-4764-9cfb-5331013fc27d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:42.931701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5235e-ea73-406d-a98e-c77bc999c02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:42.932620 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5235e-ea73-406d-a98e-c77bc999c02d admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:42.932963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5235e-ea73-406d-a98e-c77bc999c02d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:42.933382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5235e-ea73-406d-a98e-c77bc999c02d admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:42.934101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5235e-ea73-406d-a98e-c77bc999c02d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:42.937489 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5235e-ea73-406d-a98e-c77bc999c02d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:42.963637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1124/2245] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:42 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:42.969188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738171d-e52a-4918-9077-193d178ca0ff admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:42.982313 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738171d-e52a-4918-9077-193d178ca0ff admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:42.987206 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be8f60-8e7f-4ea3-a318-0dd8476acbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:42.988349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be8f60-8e7f-4ea3-a318-0dd8476acbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:42.988618 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be8f60-8e7f-4ea3-a318-0dd8476acbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:42.989054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be8f60-8e7f-4ea3-a318-0dd8476acbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:42.990144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be8f60-8e7f-4ea3-a318-0dd8476acbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:42.992157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be8f60-8e7f-4ea3-a318-0dd8476acbe9 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:43.005703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1122/2246] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:43.009243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5235e-ea73-406d-a98e-c77bc999c02d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:43.023540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5235e-ea73-406d-a98e-c77bc999c02d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:43.029085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698207af-440b-4a98-85aa-beccf77de4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:43.029795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698207af-440b-4a98-85aa-beccf77de4cc admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:43.030360 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698207af-440b-4a98-85aa-beccf77de4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:43.030704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698207af-440b-4a98-85aa-beccf77de4cc admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:43.031323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698207af-440b-4a98-85aa-beccf77de4cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:43.033009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698207af-440b-4a98-85aa-beccf77de4cc admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:43.048321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1125/2247] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:43.054734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be8f60-8e7f-4ea3-a318-0dd8476acbe9 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:43.068943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be8f60-8e7f-4ea3-a318-0dd8476acbe9 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:43.081208 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975874de-813f-4890-93ae-5232deef986d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:43.082792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975874de-813f-4890-93ae-5232deef986d admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:43.083569 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975874de-813f-4890-93ae-5232deef986d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:43.084045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975874de-813f-4890-93ae-5232deef986d admin admin] PATH_INFO: `/v3/projects/e5891046f80542a0945ae1e7cc5ed42a/users/4336f5a0781a49ce87a206f21a0ce2b7/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:43.087747 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:43.108006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975874de-813f-4890-93ae-5232deef986d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5891046f80542a0945ae1e7cc5ed42a, user_id=4336f5a0781a49ce87a206f21a0ce2b7, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:43.111510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975874de-813f-4890-93ae-5232deef986d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:43.139528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1123/2248] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:43 2022] PUT /identity/v3/projects/e5891046f80542a0945ae1e7cc5ed42a/users/4336f5a0781a49ce87a206f21a0ce2b7/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:43.158745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc0fb9-8ef9-495a-b3d4-7997a0ee6908 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:43.159319 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc0fb9-8ef9-495a-b3d4-7997a0ee6908 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:43.159756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc0fb9-8ef9-495a-b3d4-7997a0ee6908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:43.208242 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f0fc0fb9-8ef9-495a-b3d4-7997a0ee6908 None None] MFA Rules not processed for user `4336f5a0781a49ce87a206f21a0ce2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:43.210244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f0fc0fb9-8ef9-495a-b3d4-7997a0ee6908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:43.345668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975874de-813f-4890-93ae-5232deef986d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:43.346470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1126/2249] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:43.559863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975874de-813f-4890-93ae-5232deef986d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:43.567807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0583a3-ca25-4cb5-a9fa-20a8af0ecbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:43.569040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0583a3-ca25-4cb5-a9fa-20a8af0ecbdb admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:43.569040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0583a3-ca25-4cb5-a9fa-20a8af0ecbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:43.569405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0583a3-ca25-4cb5-a9fa-20a8af0ecbdb admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:43.570188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0583a3-ca25-4cb5-a9fa-20a8af0ecbdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:43.572658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0583a3-ca25-4cb5-a9fa-20a8af0ecbdb admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:43.649607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1124/2250] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:43 2022] POST /identity/v3/projects => generated 383 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:43.656586 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc0fb9-8ef9-495a-b3d4-7997a0ee6908 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:43.675463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc0fb9-8ef9-495a-b3d4-7997a0ee6908 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:43.776252 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:43.844024 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c0455-d623-45cf-af43-382117314f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:43.846143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c0455-d623-45cf-af43-382117314f78 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:43.846709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c0455-d623-45cf-af43-382117314f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:43.847386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c0455-d623-45cf-af43-382117314f78 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:43.849094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c0455-d623-45cf-af43-382117314f78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:43.854868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c0455-d623-45cf-af43-382117314f78 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:43.939420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1127/2251] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:43 2022] POST /identity/v3/users => generated 292 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:43.944022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0583a3-ca25-4cb5-a9fa-20a8af0ecbdb admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:43.971748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0583a3-ca25-4cb5-a9fa-20a8af0ecbdb admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:43.990401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97852c04-0c79-471e-afa7-9c59e8974741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:43.992490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97852c04-0c79-471e-afa7-9c59e8974741 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:43.993110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97852c04-0c79-471e-afa7-9c59e8974741 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:43.995096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97852c04-0c79-471e-afa7-9c59e8974741 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:43.997682 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97852c04-0c79-471e-afa7-9c59e8974741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:44.001201 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97852c04-0c79-471e-afa7-9c59e8974741 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:44.058901 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1125/2252] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:44.069196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c0455-d623-45cf-af43-382117314f78 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:44.092062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c0455-d623-45cf-af43-382117314f78 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:44.101978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333169b-f572-4eec-869e-c2a912602f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:44.103177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333169b-f572-4eec-869e-c2a912602f78 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:44.103681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333169b-f572-4eec-869e-c2a912602f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:44.104336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333169b-f572-4eec-869e-c2a912602f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:44.105479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333169b-f572-4eec-869e-c2a912602f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:44.108592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333169b-f572-4eec-869e-c2a912602f78 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:44.128169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1128/2253] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:44.129484 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97852c04-0c79-471e-afa7-9c59e8974741 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:44.147880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97852c04-0c79-471e-afa7-9c59e8974741 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:44.152703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b85fba-55d4-4266-96d1-c20bd1363a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:44.153388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b85fba-55d4-4266-96d1-c20bd1363a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:44.153671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b85fba-55d4-4266-96d1-c20bd1363a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:44.153948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b85fba-55d4-4266-96d1-c20bd1363a9d admin admin] PATH_INFO: `/v3/projects/8240d15446cf481e91b6b03c2e8da074/users/22f9a584f8384e3aa4e0d8f23a776d4b/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:44.172681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b85fba-55d4-4266-96d1-c20bd1363a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8240d15446cf481e91b6b03c2e8da074, user_id=22f9a584f8384e3aa4e0d8f23a776d4b, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:44.175670 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b85fba-55d4-4266-96d1-c20bd1363a9d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:44.226916 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1126/2254] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:44 2022] PUT /identity/v3/projects/8240d15446cf481e91b6b03c2e8da074/users/22f9a584f8384e3aa4e0d8f23a776d4b/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:44.240978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53eef34-d490-4c51-b4ca-03de50e0003c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:44.242037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53eef34-d490-4c51-b4ca-03de50e0003c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:44.243051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53eef34-d490-4c51-b4ca-03de50e0003c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:44.294639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-d53eef34-d490-4c51-b4ca-03de50e0003c None None] MFA Rules not processed for user `22f9a584f8384e3aa4e0d8f23a776d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:44.296371 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d53eef34-d490-4c51-b4ca-03de50e0003c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:44.410863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1129/2255] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:44.416032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b85fba-55d4-4266-96d1-c20bd1363a9d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:44.546655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b85fba-55d4-4266-96d1-c20bd1363a9d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:44.556783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570a7ff-d2ba-4a48-a44e-25cdbcbe96f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:44.557836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570a7ff-d2ba-4a48-a44e-25cdbcbe96f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:44.558323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570a7ff-d2ba-4a48-a44e-25cdbcbe96f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:44.559577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570a7ff-d2ba-4a48-a44e-25cdbcbe96f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:44.562023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570a7ff-d2ba-4a48-a44e-25cdbcbe96f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:44.568832 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570a7ff-d2ba-4a48-a44e-25cdbcbe96f4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:44.634280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1127/2256] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:44 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:44.635735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53eef34-d490-4c51-b4ca-03de50e0003c None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:44.654495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53eef34-d490-4c51-b4ca-03de50e0003c None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:44.774195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e75771-6d0f-42ae-a4b0-e959a6e4b4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:44.775853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e75771-6d0f-42ae-a4b0-e959a6e4b4e6 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:44.776353 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e75771-6d0f-42ae-a4b0-e959a6e4b4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:44.777643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e75771-6d0f-42ae-a4b0-e959a6e4b4e6 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:44.779251 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e75771-6d0f-42ae-a4b0-e959a6e4b4e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:44.783596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e75771-6d0f-42ae-a4b0-e959a6e4b4e6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:44.825360 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1130/2257] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:44 2022] POST /identity/v3/users => generated 299 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:44.838662 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570a7ff-d2ba-4a48-a44e-25cdbcbe96f4 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:44.855689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570a7ff-d2ba-4a48-a44e-25cdbcbe96f4 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:44.865690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89b698-d656-45d8-aec2-bcbe696e9433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:44.866403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89b698-d656-45d8-aec2-bcbe696e9433 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:44.866747 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89b698-d656-45d8-aec2-bcbe696e9433 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:44.867071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89b698-d656-45d8-aec2-bcbe696e9433 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:44.867987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89b698-d656-45d8-aec2-bcbe696e9433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:44.871974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89b698-d656-45d8-aec2-bcbe696e9433 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:44.887845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1128/2258] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:44.892958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e75771-6d0f-42ae-a4b0-e959a6e4b4e6 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:44.916244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e75771-6d0f-42ae-a4b0-e959a6e4b4e6 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:44.938117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab61333-995a-445d-9f5d-472314b9057f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:44.939172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab61333-995a-445d-9f5d-472314b9057f admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:44.939453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab61333-995a-445d-9f5d-472314b9057f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:44.939719 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab61333-995a-445d-9f5d-472314b9057f admin admin] PATH_INFO: `/v3/projects/a7e1af1cf7be44acb7b25af5a2c12b3b/users/3546a3e2b1e446dfab092a476d221183/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:44.959173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab61333-995a-445d-9f5d-472314b9057f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e1af1cf7be44acb7b25af5a2c12b3b, user_id=3546a3e2b1e446dfab092a476d221183, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:44.961969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab61333-995a-445d-9f5d-472314b9057f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:45.003612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1131/2259] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:44 2022] PUT /identity/v3/projects/a7e1af1cf7be44acb7b25af5a2c12b3b/users/3546a3e2b1e446dfab092a476d221183/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:45.009583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89b698-d656-45d8-aec2-bcbe696e9433 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:45.029186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89b698-d656-45d8-aec2-bcbe696e9433 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:45.036334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:45.127941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d37e7-dc7d-45fb-a9a6-b5cdb60e00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:45.129763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d37e7-dc7d-45fb-a9a6-b5cdb60e00bd admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:45.130270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d37e7-dc7d-45fb-a9a6-b5cdb60e00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:45.130609 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d37e7-dc7d-45fb-a9a6-b5cdb60e00bd admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:45.131870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d37e7-dc7d-45fb-a9a6-b5cdb60e00bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:45.134213 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d37e7-dc7d-45fb-a9a6-b5cdb60e00bd admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:45.153145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1129/2260] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:50:45.154534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab61333-995a-445d-9f5d-472314b9057f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:45.181258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab61333-995a-445d-9f5d-472314b9057f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:45.189925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ba18b-9ce0-4b55-8e55-5047795802d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:45.192827 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ba18b-9ce0-4b55-8e55-5047795802d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:45.193570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ba18b-9ce0-4b55-8e55-5047795802d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:45.194027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ba18b-9ce0-4b55-8e55-5047795802d9 admin admin] PATH_INFO: `/v3/projects/a7e1af1cf7be44acb7b25af5a2c12b3b/users/3546a3e2b1e446dfab092a476d221183/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:45.197956 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ba18b-9ce0-4b55-8e55-5047795802d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e1af1cf7be44acb7b25af5a2c12b3b, user_id=3546a3e2b1e446dfab092a476d221183, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:45.203296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ba18b-9ce0-4b55-8e55-5047795802d9 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:45.208768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:45.229406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1132/2261] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:45 2022] PUT /identity/v3/projects/a7e1af1cf7be44acb7b25af5a2c12b3b/users/3546a3e2b1e446dfab092a476d221183/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:45.246086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659092f7-298e-461d-9b32-2481f458283f None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:45.246910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659092f7-298e-461d-9b32-2481f458283f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:45.247505 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659092f7-298e-461d-9b32-2481f458283f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:45.307968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-659092f7-298e-461d-9b32-2481f458283f None None] MFA Rules not processed for user `3546a3e2b1e446dfab092a476d221183`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:45.309252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-659092f7-298e-461d-9b32-2481f458283f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:45.466348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1130/2262] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:45.482649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ba18b-9ce0-4b55-8e55-5047795802d9 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:45.492066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c83ba18b-9ce0-4b55-8e55-5047795802d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:45.615599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ba18b-9ce0-4b55-8e55-5047795802d9 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:45.623452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b13f39-329a-4157-b35c-86c69be545e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:45.624512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b13f39-329a-4157-b35c-86c69be545e7 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:45.625010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b13f39-329a-4157-b35c-86c69be545e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:45.625503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b13f39-329a-4157-b35c-86c69be545e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:45.766983 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b13f39-329a-4157-b35c-86c69be545e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:45.768528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b13f39-329a-4157-b35c-86c69be545e7 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:45.796140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1133/2263] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:46.813173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e592e12-d9fc-4803-bb9b-c8cf89754230 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:46.813729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e592e12-d9fc-4803-bb9b-c8cf89754230 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:46.814173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e592e12-d9fc-4803-bb9b-c8cf89754230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:46.871559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-3e592e12-d9fc-4803-bb9b-c8cf89754230 None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:46.873183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3e592e12-d9fc-4803-bb9b-c8cf89754230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:46.958600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1131/2264] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:50:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:47.039239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b13f39-329a-4157-b35c-86c69be545e7 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:47.040800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-85b13f39-329a-4157-b35c-86c69be545e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:47.135209 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b13f39-329a-4157-b35c-86c69be545e7 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:47.148919 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82852795-1d36-4260-a1d5-61175837fdd1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:47.149558 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82852795-1d36-4260-a1d5-61175837fdd1 service swift] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:47.149833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82852795-1d36-4260-a1d5-61175837fdd1 service swift] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:47.150098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82852795-1d36-4260-a1d5-61175837fdd1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:47.159808 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82852795-1d36-4260-a1d5-61175837fdd1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:47.161423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82852795-1d36-4260-a1d5-61175837fdd1 service swift] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:47.172516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1134/2265] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:50:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:50:48.011216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e592e12-d9fc-4803-bb9b-c8cf89754230 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:48.024585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e592e12-d9fc-4803-bb9b-c8cf89754230 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:48.038425 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a984702-14dc-464d-8f48-2b007aaa1eca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:48.039473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a984702-14dc-464d-8f48-2b007aaa1eca service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:48.040211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a984702-14dc-464d-8f48-2b007aaa1eca service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:48.040910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a984702-14dc-464d-8f48-2b007aaa1eca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:48.115716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:48.185802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a984702-14dc-464d-8f48-2b007aaa1eca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:48.189004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a984702-14dc-464d-8f48-2b007aaa1eca service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:48.205154 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f43f4-53da-451c-ad70-0f91b0f30f33 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:48.206540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f43f4-53da-451c-ad70-0f91b0f30f33 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:48.206540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f43f4-53da-451c-ad70-0f91b0f30f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:48.208215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1132/2266] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:50:48.231001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f86f43f4-53da-451c-ad70-0f91b0f30f33 None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:48.232364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f86f43f4-53da-451c-ad70-0f91b0f30f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:48.254415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1135/2267] 10.222.0.36 () {58 vars in 967 bytes} [Wed Aug 31 09:50:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:48.288655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a984702-14dc-464d-8f48-2b007aaa1eca service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:48.307968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a984702-14dc-464d-8f48-2b007aaa1eca service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:48.316632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c866856-6ebd-4666-8d3f-cdb444b7c391 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:48.317484 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c866856-6ebd-4666-8d3f-cdb444b7c391 service swift] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:48.318474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c866856-6ebd-4666-8d3f-cdb444b7c391 service swift] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:48.318853 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c866856-6ebd-4666-8d3f-cdb444b7c391 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:48.330580 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c866856-6ebd-4666-8d3f-cdb444b7c391 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:48.332649 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c866856-6ebd-4666-8d3f-cdb444b7c391 service swift] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:48.348624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1133/2268] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:50:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:50:48.823259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20b61b-e4a4-43b6-aae1-fa0549fa09ad None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:48.824282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20b61b-e4a4-43b6-aae1-fa0549fa09ad None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:48.825116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20b61b-e4a4-43b6-aae1-fa0549fa09ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:48.868096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-8a20b61b-e4a4-43b6-aae1-fa0549fa09ad None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:48.870101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-8a20b61b-e4a4-43b6-aae1-fa0549fa09ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:48.884367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1136/2269] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:50:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:48.905063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c866856-6ebd-4666-8d3f-cdb444b7c391 service swift] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:48.929725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c866856-6ebd-4666-8d3f-cdb444b7c391 service swift] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:48.942681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb440af4-7cb7-4ae5-bf0e-c59ef6f6ac1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:48.943996 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb440af4-7cb7-4ae5-bf0e-c59ef6f6ac1d service swift] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:48.944692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb440af4-7cb7-4ae5-bf0e-c59ef6f6ac1d service swift] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:48.945282 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb440af4-7cb7-4ae5-bf0e-c59ef6f6ac1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:48.966666 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb440af4-7cb7-4ae5-bf0e-c59ef6f6ac1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:48.970037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb440af4-7cb7-4ae5-bf0e-c59ef6f6ac1d service swift] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:48.987382 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1134/2270] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:50:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:50:49.306666 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20b61b-e4a4-43b6-aae1-fa0549fa09ad None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:49.309159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-8a20b61b-e4a4-43b6-aae1-fa0549fa09ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:49.331623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20b61b-e4a4-43b6-aae1-fa0549fa09ad None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:49.337836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eba6c-69e7-468a-8eba-248577d37753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:49.338595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eba6c-69e7-468a-8eba-248577d37753 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:49.338916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eba6c-69e7-468a-8eba-248577d37753 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:49.339280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eba6c-69e7-468a-8eba-248577d37753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:49.354405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40eba6c-69e7-468a-8eba-248577d37753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:49.356863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40eba6c-69e7-468a-8eba-248577d37753 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:49.375498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1137/2271] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:49.551366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837aaa0-db88-4890-813d-b6656f755b5f None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:49.552842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837aaa0-db88-4890-813d-b6656f755b5f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:49.553383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837aaa0-db88-4890-813d-b6656f755b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:49.592704 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-3837aaa0-db88-4890-813d-b6656f755b5f None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:49.594991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3837aaa0-db88-4890-813d-b6656f755b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:49.615883 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1135/2272] 10.222.0.36 () {58 vars in 967 bytes} [Wed Aug 31 09:50:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:49.637985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eba6c-69e7-468a-8eba-248577d37753 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:49.655645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eba6c-69e7-468a-8eba-248577d37753 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:49.666603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4dff5-cd36-4504-b5db-e8f3ba000875 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:49.667266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4dff5-cd36-4504-b5db-e8f3ba000875 service swift] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:49.667936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4dff5-cd36-4504-b5db-e8f3ba000875 service swift] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:49.668221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4dff5-cd36-4504-b5db-e8f3ba000875 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:49.680398 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4dff5-cd36-4504-b5db-e8f3ba000875 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:49.682737 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4dff5-cd36-4504-b5db-e8f3ba000875 service swift] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:49.688780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:49.696961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1138/2273] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:50:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:50:52.345458 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837aaa0-db88-4890-813d-b6656f755b5f None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:52.351744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3837aaa0-db88-4890-813d-b6656f755b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:52.541772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837aaa0-db88-4890-813d-b6656f755b5f None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:52.552143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3ab2a-947b-4ab1-b7b1-3abb9156379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:52.553248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3ab2a-947b-4ab1-b7b1-3abb9156379c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:52.553747 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3ab2a-947b-4ab1-b7b1-3abb9156379c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:52.554276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3ab2a-947b-4ab1-b7b1-3abb9156379c admin admin] PATH_INFO: `/v3/users/646da13f9db94b0fa2302d6b45774e54` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:52.556641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3ab2a-947b-4ab1-b7b1-3abb9156379c admin admin] RBAC: Authorizing `identity:delete_user(user_id=646da13f9db94b0fa2302d6b45774e54)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:52.560842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3ab2a-947b-4ab1-b7b1-3abb9156379c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:52.715531 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-55a3ab2a-947b-4ab1-b7b1-3abb9156379c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646da13f9db94b0fa2302d6b45774e54'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:52.723484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4dff5-cd36-4504-b5db-e8f3ba000875 service swift] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:52.726095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-cce4dff5-cd36-4504-b5db-e8f3ba000875 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:52.759192 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-55a3ab2a-947b-4ab1-b7b1-3abb9156379c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646da13f9db94b0fa2302d6b45774e54'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:52.760470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-55a3ab2a-947b-4ab1-b7b1-3abb9156379c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646da13f9db94b0fa2302d6b45774e54'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:52.761239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-55a3ab2a-947b-4ab1-b7b1-3abb9156379c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646da13f9db94b0fa2302d6b45774e54'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:52.776823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-55a3ab2a-947b-4ab1-b7b1-3abb9156379c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646da13f9db94b0fa2302d6b45774e54'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:52.807822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4dff5-cd36-4504-b5db-e8f3ba000875 service swift] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:52.811863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1136/2274] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:52 2022] DELETE /identity/v3/users/646da13f9db94b0fa2302d6b45774e54 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:52.818700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3b010-e6e3-4bf2-80eb-a7e47d49aee3 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:52.819385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3b010-e6e3-4bf2-80eb-a7e47d49aee3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:52.820979 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3b010-e6e3-4bf2-80eb-a7e47d49aee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:52.824188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:52.825903 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:52.826827 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:52.827596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] PATH_INFO: `/v3/users/bf45320c49bd443388f5af0e3871bc2e` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:52.830137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf45320c49bd443388f5af0e3871bc2e)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:52.836411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:52.870342 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-3ec3b010-e6e3-4bf2-80eb-a7e47d49aee3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:52.873083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3ec3b010-e6e3-4bf2-80eb-a7e47d49aee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:52.916048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:52.980133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf45320c49bd443388f5af0e3871bc2e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:53.000442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf45320c49bd443388f5af0e3871bc2e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:53.001664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf45320c49bd443388f5af0e3871bc2e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:53.028665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf45320c49bd443388f5af0e3871bc2e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:53.030162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf45320c49bd443388f5af0e3871bc2e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:53.049864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1139/2275] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:52 2022] DELETE /identity/v3/users/bf45320c49bd443388f5af0e3871bc2e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:53.066909 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1137/2276] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:53.068980 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:53.069736 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87606c6-0bc8-4421-8b55-c12c3bbf3a39 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:53.070118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87606c6-0bc8-4421-8b55-c12c3bbf3a39 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:53.070483 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87606c6-0bc8-4421-8b55-c12c3bbf3a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:53.082019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:53.105742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e87606c6-0bc8-4421-8b55-c12c3bbf3a39 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:53.108841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e87606c6-0bc8-4421-8b55-c12c3bbf3a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:53.202888 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b029a-5256-498d-ba8a-62bbdbc88516 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:53.208331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d43f9-1c59-411c-8588-962cfd6178dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:53.209259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d43f9-1c59-411c-8588-962cfd6178dc service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:53.209618 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d43f9-1c59-411c-8588-962cfd6178dc service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:53.209900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d43f9-1c59-411c-8588-962cfd6178dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:53.261456 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1138/2277] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:53.275583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87606c6-0bc8-4421-8b55-c12c3bbf3a39 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:53.288454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87606c6-0bc8-4421-8b55-c12c3bbf3a39 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:53.293845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bea990-39f2-4da4-8962-477254f9280b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:53.294613 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bea990-39f2-4da4-8962-477254f9280b service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:53.294951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bea990-39f2-4da4-8962-477254f9280b service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:53.295457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bea990-39f2-4da4-8962-477254f9280b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:53.314626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bea990-39f2-4da4-8962-477254f9280b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:53.317869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bea990-39f2-4da4-8962-477254f9280b service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:53.331267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1139/2278] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:50:53.348989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d43f9-1c59-411c-8588-962cfd6178dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:53.352603 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d43f9-1c59-411c-8588-962cfd6178dc service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:53.372527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1140/2279] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:53.947506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bea990-39f2-4da4-8962-477254f9280b service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:53.953728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c4bea990-39f2-4da4-8962-477254f9280b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:53.994622 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bea990-39f2-4da4-8962-477254f9280b service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:54.002817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:54.004756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.005267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.005975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] PATH_INFO: `/v3/projects/7d45013dccd34a8994a4b25c6ac0f2bf` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.008189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d45013dccd34a8994a4b25c6ac0f2bf)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:54.011304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:54.134493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d45013dccd34a8994a4b25c6ac0f2bf', 'request_id': 'req-82c354d5-17fa-42ab-8cb8-c2da722e6931'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.169269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d45013dccd34a8994a4b25c6ac0f2bf', 'request_id': 'req-82c354d5-17fa-42ab-8cb8-c2da722e6931'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.170835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d43f9-1c59-411c-8588-962cfd6178dc service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:54.176129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-007d43f9-1c59-411c-8588-962cfd6178dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:54.189351 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d45013dccd34a8994a4b25c6ac0f2bf', 'request_id': 'req-82c354d5-17fa-42ab-8cb8-c2da722e6931'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.207966 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d43f9-1c59-411c-8588-962cfd6178dc service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:54.234215 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] The token cache is being invalidate because project 7d45013dccd34a8994a4b25c6ac0f2bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:54.236007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.239319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.247186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1140/2280] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:53 2022] DELETE /identity/v3/projects/7d45013dccd34a8994a4b25c6ac0f2bf => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:54.247186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:54.252442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:54.273932 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:54.335772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:54.407788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c354d5-17fa-42ab-8cb8-c2da722e6931 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:54.417745 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975e7e0-d944-4aea-a919-4b6dbabfd02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:54.418795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975e7e0-d944-4aea-a919-4b6dbabfd02e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.418976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:54.419347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975e7e0-d944-4aea-a919-4b6dbabfd02e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.419811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975e7e0-d944-4aea-a919-4b6dbabfd02e admin admin] PATH_INFO: `/v3/users/4336f5a0781a49ce87a206f21a0ce2b7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.419982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.420359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.420748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9 admin admin] PATH_INFO: `/v3/projects/c88af70f29db4015a6fce06089debe5c` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.422449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c88af70f29db4015a6fce06089debe5c)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:54.422853 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975e7e0-d944-4aea-a919-4b6dbabfd02e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4336f5a0781a49ce87a206f21a0ce2b7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:54.426601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975e7e0-d944-4aea-a919-4b6dbabfd02e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:54.427324 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:54.521568 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c88af70f29db4015a6fce06089debe5c', 'request_id': 'req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.529891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c88af70f29db4015a6fce06089debe5c', 'request_id': 'req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.541197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c88af70f29db4015a6fce06089debe5c', 'request_id': 'req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.553434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d975e7e0-d944-4aea-a919-4b6dbabfd02e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4336f5a0781a49ce87a206f21a0ce2b7'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.559914 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9 admin admin] The token cache is being invalidate because project c88af70f29db4015a6fce06089debe5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:54.560356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.561577 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e364cac0-dcb9-4225-9c2e-354d9a6fd6b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.564087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1141/2281] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:54 2022] DELETE /identity/v3/projects/c88af70f29db4015a6fce06089debe5c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:54.573003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaa6f9-c449-476e-ac88-885a6506565c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.573314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaa6f9-c449-476e-ac88-885a6506565c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.573651 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaa6f9-c449-476e-ac88-885a6506565c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.589971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d975e7e0-d944-4aea-a919-4b6dbabfd02e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4336f5a0781a49ce87a206f21a0ce2b7'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.590821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d975e7e0-d944-4aea-a919-4b6dbabfd02e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4336f5a0781a49ce87a206f21a0ce2b7'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.591607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d975e7e0-d944-4aea-a919-4b6dbabfd02e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4336f5a0781a49ce87a206f21a0ce2b7'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.602234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-d975e7e0-d944-4aea-a919-4b6dbabfd02e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4336f5a0781a49ce87a206f21a0ce2b7'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:54.602707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-c1eaa6f9-c449-476e-ac88-885a6506565c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:54.603940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c1eaa6f9-c449-476e-ac88-885a6506565c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:54.620218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1141/2282] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:54 2022] DELETE /identity/v3/users/4336f5a0781a49ce87a206f21a0ce2b7 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:54.627581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d10d5-b9b0-4323-b815-a8514fb8ec46 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.628021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d10d5-b9b0-4323-b815-a8514fb8ec46 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.628942 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d10d5-b9b0-4323-b815-a8514fb8ec46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.652387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-1c8d10d5-b9b0-4323-b815-a8514fb8ec46 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:54.653683 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1c8d10d5-b9b0-4323-b815-a8514fb8ec46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:54.717171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1142/2283] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:54.720922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620ceb7-5dd1-4ab2-a9e3-e3bf8307707f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.721492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620ceb7-5dd1-4ab2-a9e3-e3bf8307707f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.722989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620ceb7-5dd1-4ab2-a9e3-e3bf8307707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.752050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-5620ceb7-5dd1-4ab2-a9e3-e3bf8307707f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:54.755077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1142/2284] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:54.755781 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5620ceb7-5dd1-4ab2-a9e3-e3bf8307707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:54.767062 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d10d5-b9b0-4323-b815-a8514fb8ec46 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:54.768541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1c8d10d5-b9b0-4323-b815-a8514fb8ec46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:54.772747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1143/2285] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:54.781341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620ceb7-5dd1-4ab2-a9e3-e3bf8307707f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:54.789171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620ceb7-5dd1-4ab2-a9e3-e3bf8307707f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:54.794893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ecf95-2965-46fb-8116-d578635ab35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:54.795614 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ecf95-2965-46fb-8116-d578635ab35a admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.795899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ecf95-2965-46fb-8116-d578635ab35a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.796171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ecf95-2965-46fb-8116-d578635ab35a admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.796837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ecf95-2965-46fb-8116-d578635ab35a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:54.797142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ecf95-2965-46fb-8116-d578635ab35a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:54.798788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ecf95-2965-46fb-8116-d578635ab35a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:54.822469 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1144/2286] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:50:54.823702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ecf95-2965-46fb-8116-d578635ab35a admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:54.835914 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ecf95-2965-46fb-8116-d578635ab35a admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:54.844143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f2a50-cb97-4e33-aefd-b1c37f82161b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:54.845146 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f2a50-cb97-4e33-aefd-b1c37f82161b admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.845495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f2a50-cb97-4e33-aefd-b1c37f82161b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.845894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f2a50-cb97-4e33-aefd-b1c37f82161b admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.846630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f2a50-cb97-4e33-aefd-b1c37f82161b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:54.849399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f2a50-cb97-4e33-aefd-b1c37f82161b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:54.877563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:54.886043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d10d5-b9b0-4323-b815-a8514fb8ec46 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:54.896160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d401-a728-4ad6-908a-f657291c2ddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:54.897267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d401-a728-4ad6-908a-f657291c2ddd service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.902879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d401-a728-4ad6-908a-f657291c2ddd service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.903115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1145/2287] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:54 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:54.903274 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d401-a728-4ad6-908a-f657291c2ddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.904520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f2a50-cb97-4e33-aefd-b1c37f82161b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:54.920419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:54.923197 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1d401-a728-4ad6-908a-f657291c2ddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:54.925805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1d401-a728-4ad6-908a-f657291c2ddd service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:54.947050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1143/2288] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:50:55.006560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f2a50-cb97-4e33-aefd-b1c37f82161b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:55.015825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f30daf-223f-4246-ac96-ecf4e993ec19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:55.016931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f30daf-223f-4246-ac96-ecf4e993ec19 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:55.017445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f30daf-223f-4246-ac96-ecf4e993ec19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:55.017957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f30daf-223f-4246-ac96-ecf4e993ec19 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:55.019756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f30daf-223f-4246-ac96-ecf4e993ec19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:55.023935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f30daf-223f-4246-ac96-ecf4e993ec19 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:55.066686 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1146/2289] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:54 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:55.068485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d401-a728-4ad6-908a-f657291c2ddd service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:55.080701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d401-a728-4ad6-908a-f657291c2ddd service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:55.089665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dfdb7-10fe-430e-a37d-4a0138dea553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:55.090845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dfdb7-10fe-430e-a37d-4a0138dea553 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:55.091492 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dfdb7-10fe-430e-a37d-4a0138dea553 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:55.092068 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dfdb7-10fe-430e-a37d-4a0138dea553 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:55.093347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dfdb7-10fe-430e-a37d-4a0138dea553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:55.096627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dfdb7-10fe-430e-a37d-4a0138dea553 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:55.124225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1144/2290] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:55.129537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f30daf-223f-4246-ac96-ecf4e993ec19 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:55.142726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f30daf-223f-4246-ac96-ecf4e993ec19 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:55.156115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ef52b-7c86-4337-903d-e5ea62670c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:55.157416 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ef52b-7c86-4337-903d-e5ea62670c74 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:55.158792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ef52b-7c86-4337-903d-e5ea62670c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:55.159731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ef52b-7c86-4337-903d-e5ea62670c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:55.160775 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ef52b-7c86-4337-903d-e5ea62670c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:55.162819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ef52b-7c86-4337-903d-e5ea62670c74 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:55.182525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1147/2291] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:55.188105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dfdb7-10fe-430e-a37d-4a0138dea553 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:55.226171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dfdb7-10fe-430e-a37d-4a0138dea553 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:55.234120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde32f0-4e6a-4014-a1f5-f21986c775ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:55.235291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde32f0-4e6a-4014-a1f5-f21986c775ff admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:55.235743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde32f0-4e6a-4014-a1f5-f21986c775ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:55.236151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde32f0-4e6a-4014-a1f5-f21986c775ff admin admin] PATH_INFO: `/v3/projects/9f1c2adbca494a53a24033b674c2762c/users/893e06c5ea99497fae2847399b7a1a3c/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:55.269722 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde32f0-4e6a-4014-a1f5-f21986c775ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1c2adbca494a53a24033b674c2762c, user_id=893e06c5ea99497fae2847399b7a1a3c, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:55.274757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde32f0-4e6a-4014-a1f5-f21986c775ff admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:55.311671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1145/2292] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:55 2022] PUT /identity/v3/projects/9f1c2adbca494a53a24033b674c2762c/users/893e06c5ea99497fae2847399b7a1a3c/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:55.319154 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b3fde-93d5-48f4-96df-0b4aebdad154 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:55.319587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b3fde-93d5-48f4-96df-0b4aebdad154 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:55.320058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b3fde-93d5-48f4-96df-0b4aebdad154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:55.358314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde32f0-4e6a-4014-a1f5-f21986c775ff admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:55.361250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1cde32f0-4e6a-4014-a1f5-f21986c775ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:55.369666 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-181b3fde-93d5-48f4-96df-0b4aebdad154 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:55.373445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-181b3fde-93d5-48f4-96df-0b4aebdad154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:55.510707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde32f0-4e6a-4014-a1f5-f21986c775ff admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:55.516426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:55.517203 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:55.517574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:55.517940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] PATH_INFO: `/v3/projects/e5891046f80542a0945ae1e7cc5ed42a` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:55.520110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5891046f80542a0945ae1e7cc5ed42a)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:55.522297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:55.522952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1148/2293] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:55.537365 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b3fde-93d5-48f4-96df-0b4aebdad154 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:55.539407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-181b3fde-93d5-48f4-96df-0b4aebdad154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:55.611790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5891046f80542a0945ae1e7cc5ed42a', 'request_id': 'req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:55.626105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5891046f80542a0945ae1e7cc5ed42a', 'request_id': 'req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:55.626904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b3fde-93d5-48f4-96df-0b4aebdad154 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:55.638127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5891046f80542a0945ae1e7cc5ed42a', 'request_id': 'req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:55.661475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] The token cache is being invalidate because project e5891046f80542a0945ae1e7cc5ed42a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:55.661956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:55.663086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:55.665507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1146/2294] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:55 2022] DELETE /identity/v3/projects/e5891046f80542a0945ae1e7cc5ed42a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:55.670950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:55.673224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:55.714184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:55.725432 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc55a07-43e4-4ebb-9e00-7653fe96eb46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:55.726950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc55a07-43e4-4ebb-9e00-7653fe96eb46 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:55.727693 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc55a07-43e4-4ebb-9e00-7653fe96eb46 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:55.728382 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc55a07-43e4-4ebb-9e00-7653fe96eb46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:55.730659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-bfc55a07-43e4-4ebb-9e00-7653fe96eb46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:55.740084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:55.768181 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7e200-ce52-485c-9bb2-0d1e5dc3443e admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:55.777273 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:55.778541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:55.779041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:55.779706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] PATH_INFO: `/v3/users/22f9a584f8384e3aa4e0d8f23a776d4b` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:55.782029 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22f9a584f8384e3aa4e0d8f23a776d4b)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:55.786743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:55.864515 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc55a07-43e4-4ebb-9e00-7653fe96eb46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:55.866951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc55a07-43e4-4ebb-9e00-7653fe96eb46 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:55.878902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22f9a584f8384e3aa4e0d8f23a776d4b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:55.881692 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1149/2295] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:55.902216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22f9a584f8384e3aa4e0d8f23a776d4b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:55.903039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22f9a584f8384e3aa4e0d8f23a776d4b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:55.903950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22f9a584f8384e3aa4e0d8f23a776d4b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:55.920640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22f9a584f8384e3aa4e0d8f23a776d4b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:55.942605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1147/2296] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:55 2022] DELETE /identity/v3/users/22f9a584f8384e3aa4e0d8f23a776d4b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:56.348794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc55a07-43e4-4ebb-9e00-7653fe96eb46 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:56.368261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc55a07-43e4-4ebb-9e00-7653fe96eb46 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:56.501585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:56.507002 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:56.508803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:56.510333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] PATH_INFO: `/v3/projects/8240d15446cf481e91b6b03c2e8da074` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:56.516531 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8240d15446cf481e91b6b03c2e8da074)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:56.520245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:56.599747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8240d15446cf481e91b6b03c2e8da074', 'request_id': 'req-a1692573-ba13-4ee4-b6d8-db29605e517b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:56.610176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8240d15446cf481e91b6b03c2e8da074', 'request_id': 'req-a1692573-ba13-4ee4-b6d8-db29605e517b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:56.620353 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8240d15446cf481e91b6b03c2e8da074', 'request_id': 'req-a1692573-ba13-4ee4-b6d8-db29605e517b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:56.636826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] The token cache is being invalidate because project 8240d15446cf481e91b6b03c2e8da074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:56.637157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:56.637936 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:56.642540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1150/2297] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:56 2022] DELETE /identity/v3/projects/8240d15446cf481e91b6b03c2e8da074 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:56.643918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:56.645563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:56.663189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1e77c-b1b7-4016-8e91-443a9e582313 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:56.776442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697edd6-426a-4cf9-b9e2-8bff4b2b2563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:56.777092 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697edd6-426a-4cf9-b9e2-8bff4b2b2563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:56.777495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697edd6-426a-4cf9-b9e2-8bff4b2b2563 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:56.777752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697edd6-426a-4cf9-b9e2-8bff4b2b2563 admin admin] PATH_INFO: `/v3/users/3546a3e2b1e446dfab092a476d221183` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:56.779142 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697edd6-426a-4cf9-b9e2-8bff4b2b2563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3546a3e2b1e446dfab092a476d221183)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:56.783044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697edd6-426a-4cf9-b9e2-8bff4b2b2563 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:56.865954 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b697edd6-426a-4cf9-b9e2-8bff4b2b2563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3546a3e2b1e446dfab092a476d221183'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:56.883874 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b697edd6-426a-4cf9-b9e2-8bff4b2b2563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3546a3e2b1e446dfab092a476d221183'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:56.885020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b697edd6-426a-4cf9-b9e2-8bff4b2b2563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3546a3e2b1e446dfab092a476d221183'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:56.885678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b697edd6-426a-4cf9-b9e2-8bff4b2b2563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3546a3e2b1e446dfab092a476d221183'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:56.891986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:56.898746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b697edd6-426a-4cf9-b9e2-8bff4b2b2563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3546a3e2b1e446dfab092a476d221183'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:56.927193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1148/2298] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:56 2022] DELETE /identity/v3/users/3546a3e2b1e446dfab092a476d221183 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:57.273850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:57.276575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:57.325597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1692573-ba13-4ee4-b6d8-db29605e517b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:57.412882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:57.479285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:57.479943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:57.480405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:57.480829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] PATH_INFO: `/v3/users/02b26d233b7b440da825dfc619c90e8f` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:57.482355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b26d233b7b440da825dfc619c90e8f)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:57.485427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:57.574064 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b26d233b7b440da825dfc619c90e8f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:57.587605 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b26d233b7b440da825dfc619c90e8f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:57.589665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b26d233b7b440da825dfc619c90e8f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:57.632550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b26d233b7b440da825dfc619c90e8f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:57.634743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b26d233b7b440da825dfc619c90e8f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:57.677987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1151/2299] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:57 2022] DELETE /identity/v3/users/02b26d233b7b440da825dfc619c90e8f => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:57.684160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1691e9b-69ed-4f49-86ac-23ae395d05f3 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:57.686257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1691e9b-69ed-4f49-86ac-23ae395d05f3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:57.686687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1691e9b-69ed-4f49-86ac-23ae395d05f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:57.703631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:57.710416 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:57.749707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-c1691e9b-69ed-4f49-86ac-23ae395d05f3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:57.758769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c1691e9b-69ed-4f49-86ac-23ae395d05f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:57.941769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4915e2-287c-4493-aa9f-e849dc2bcdf1 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:57.962263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4033af-4542-4169-a8b8-0b11c039c40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:57.964611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4033af-4542-4169-a8b8-0b11c039c40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:57.965391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4033af-4542-4169-a8b8-0b11c039c40c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:57.965902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1149/2300] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:57.966740 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4033af-4542-4169-a8b8-0b11c039c40c admin admin] PATH_INFO: `/v3/projects/a7e1af1cf7be44acb7b25af5a2c12b3b` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:57.969456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4033af-4542-4169-a8b8-0b11c039c40c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7e1af1cf7be44acb7b25af5a2c12b3b)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:57.975475 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4033af-4542-4169-a8b8-0b11c039c40c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:57.992229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1691e9b-69ed-4f49-86ac-23ae395d05f3 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:57.995458 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c1691e9b-69ed-4f49-86ac-23ae395d05f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:58.135143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:58.138977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4d4033af-4542-4169-a8b8-0b11c039c40c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7e1af1cf7be44acb7b25af5a2c12b3b', 'request_id': 'req-4d4033af-4542-4169-a8b8-0b11c039c40c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:58.155895 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4d4033af-4542-4169-a8b8-0b11c039c40c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7e1af1cf7be44acb7b25af5a2c12b3b', 'request_id': 'req-4d4033af-4542-4169-a8b8-0b11c039c40c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:58.183406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4d4033af-4542-4169-a8b8-0b11c039c40c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7e1af1cf7be44acb7b25af5a2c12b3b', 'request_id': 'req-4d4033af-4542-4169-a8b8-0b11c039c40c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:58.200180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1691e9b-69ed-4f49-86ac-23ae395d05f3 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:58.237181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4d4033af-4542-4169-a8b8-0b11c039c40c admin admin] The token cache is being invalidate because project a7e1af1cf7be44acb7b25af5a2c12b3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:58.238202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4d4033af-4542-4169-a8b8-0b11c039c40c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:58.255170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4d4033af-4542-4169-a8b8-0b11c039c40c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:58.261906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1152/2301] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:57 2022] DELETE /identity/v3/projects/a7e1af1cf7be44acb7b25af5a2c12b3b => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:58.266189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6ea30-eeb6-4403-80ff-504b85626958 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:58.266704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6ea30-eeb6-4403-80ff-504b85626958 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:58.267167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6ea30-eeb6-4403-80ff-504b85626958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:58.302852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a4a6ea30-eeb6-4403-80ff-504b85626958 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:58.305446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a4a6ea30-eeb6-4403-80ff-504b85626958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:58.334257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c263435-8459-498c-8ad1-8b550ef8b3bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:58.335296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c263435-8459-498c-8ad1-8b550ef8b3bb service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:58.335997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c263435-8459-498c-8ad1-8b550ef8b3bb service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:58.336219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c263435-8459-498c-8ad1-8b550ef8b3bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:58.340096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-5c263435-8459-498c-8ad1-8b550ef8b3bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:58.469924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1153/2302] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:58.493870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:58.526964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c263435-8459-498c-8ad1-8b550ef8b3bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:58.529767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c263435-8459-498c-8ad1-8b550ef8b3bb service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:58.562853 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1150/2303] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:50:58.578389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21dbe4-b0e1-4e27-906f-0695d538aa91 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:58.579014 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21dbe4-b0e1-4e27-906f-0695d538aa91 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:58.579434 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21dbe4-b0e1-4e27-906f-0695d538aa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:58.621268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-7c21dbe4-b0e1-4e27-906f-0695d538aa91 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:58.623738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7c21dbe4-b0e1-4e27-906f-0695d538aa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:58.640710 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1154/2304] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:58.648822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c263435-8459-498c-8ad1-8b550ef8b3bb service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:58.670008 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c263435-8459-498c-8ad1-8b550ef8b3bb service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:58.693289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e495f5-a5e3-402a-9fb0-30b40b8e9c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:58.694774 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e495f5-a5e3-402a-9fb0-30b40b8e9c48 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:58.695596 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e495f5-a5e3-402a-9fb0-30b40b8e9c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:58.699290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e495f5-a5e3-402a-9fb0-30b40b8e9c48 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:58.700532 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e495f5-a5e3-402a-9fb0-30b40b8e9c48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:58.701014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e495f5-a5e3-402a-9fb0-30b40b8e9c48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:58.702978 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e495f5-a5e3-402a-9fb0-30b40b8e9c48 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:58.739147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1151/2305] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:50:58.745664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21dbe4-b0e1-4e27-906f-0695d538aa91 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:58.776528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21dbe4-b0e1-4e27-906f-0695d538aa91 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:58.789961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad04cb-e1c7-4432-84e9-f40e32b67b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:58.791157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad04cb-e1c7-4432-84e9-f40e32b67b20 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:58.791644 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad04cb-e1c7-4432-84e9-f40e32b67b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:58.792109 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad04cb-e1c7-4432-84e9-f40e32b67b20 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:58.793371 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad04cb-e1c7-4432-84e9-f40e32b67b20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:58.798881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad04cb-e1c7-4432-84e9-f40e32b67b20 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:58.866195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1155/2306] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:58 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:58.867151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e495f5-a5e3-402a-9fb0-30b40b8e9c48 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:58.888998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e495f5-a5e3-402a-9fb0-30b40b8e9c48 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:58.999058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b4b20-fc7f-48c7-93fa-ee68ea65fe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:58.999728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b4b20-fc7f-48c7-93fa-ee68ea65fe0f admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:59.000010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b4b20-fc7f-48c7-93fa-ee68ea65fe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:59.000344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b4b20-fc7f-48c7-93fa-ee68ea65fe0f admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:59.001544 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b4b20-fc7f-48c7-93fa-ee68ea65fe0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:59.004470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b4b20-fc7f-48c7-93fa-ee68ea65fe0f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:59.101217 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1152/2307] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:58 2022] POST /identity/v3/users => generated 295 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:59.102111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad04cb-e1c7-4432-84e9-f40e32b67b20 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:59.117710 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad04cb-e1c7-4432-84e9-f40e32b67b20 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:59.124338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f7d29-41b5-4c15-946d-806f06a233f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:59.127525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f7d29-41b5-4c15-946d-806f06a233f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:59.128472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f7d29-41b5-4c15-946d-806f06a233f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:59.129107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f7d29-41b5-4c15-946d-806f06a233f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:59.130613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f7d29-41b5-4c15-946d-806f06a233f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:59.134059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f7d29-41b5-4c15-946d-806f06a233f0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:59.156094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1156/2308] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:59.162907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b4b20-fc7f-48c7-93fa-ee68ea65fe0f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:59.182538 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b4b20-fc7f-48c7-93fa-ee68ea65fe0f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:59.190158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76e564-ffdd-46ea-8a2b-0d7b2d16e9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:59.191675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76e564-ffdd-46ea-8a2b-0d7b2d16e9fa admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:59.192051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76e564-ffdd-46ea-8a2b-0d7b2d16e9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:59.192492 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76e564-ffdd-46ea-8a2b-0d7b2d16e9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:59.193452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76e564-ffdd-46ea-8a2b-0d7b2d16e9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:59.196010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76e564-ffdd-46ea-8a2b-0d7b2d16e9fa admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:59.232300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1153/2309] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:59.238848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f7d29-41b5-4c15-946d-806f06a233f0 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:59.261063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f7d29-41b5-4c15-946d-806f06a233f0 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:59.266229 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f359a-1bbf-4426-bae3-de5570b77347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:59.266843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f359a-1bbf-4426-bae3-de5570b77347 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:59.267115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f359a-1bbf-4426-bae3-de5570b77347 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:59.267566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f359a-1bbf-4426-bae3-de5570b77347 admin admin] PATH_INFO: `/v3/projects/5a5de530516f49fc97f8c40fc2a64df4/users/79cef444ee0a434c9cd2ef96b7af1536/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:59.293078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f359a-1bbf-4426-bae3-de5570b77347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a5de530516f49fc97f8c40fc2a64df4, user_id=79cef444ee0a434c9cd2ef96b7af1536, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:59.295950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f359a-1bbf-4426-bae3-de5570b77347 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:59.315441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1157/2310] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:59 2022] PUT /identity/v3/projects/5a5de530516f49fc97f8c40fc2a64df4/users/79cef444ee0a434c9cd2ef96b7af1536/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:59.327463 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e95971-234a-44cd-8aaa-8d2a22098e27 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:59.328127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e95971-234a-44cd-8aaa-8d2a22098e27 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:59.328478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e95971-234a-44cd-8aaa-8d2a22098e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:59.366592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-06e95971-234a-44cd-8aaa-8d2a22098e27 None None] MFA Rules not processed for user `79cef444ee0a434c9cd2ef96b7af1536`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:59.369156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-06e95971-234a-44cd-8aaa-8d2a22098e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:59.511462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1154/2311] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:50:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:59.528806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f359a-1bbf-4426-bae3-de5570b77347 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:59.536442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b94f359a-1bbf-4426-bae3-de5570b77347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:59.561996 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e95971-234a-44cd-8aaa-8d2a22098e27 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:59.563152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-06e95971-234a-44cd-8aaa-8d2a22098e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:59.669491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f359a-1bbf-4426-bae3-de5570b77347 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:59.678200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c3aaf-c900-4147-8dfb-0046a381a295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:59.679950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c3aaf-c900-4147-8dfb-0046a381a295 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:59.681460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c3aaf-c900-4147-8dfb-0046a381a295 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:59.682497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c3aaf-c900-4147-8dfb-0046a381a295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:59.705723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c3aaf-c900-4147-8dfb-0046a381a295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:59.708193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c3aaf-c900-4147-8dfb-0046a381a295 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:59.718758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:59.723823 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1158/2312] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:59.759671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e95971-234a-44cd-8aaa-8d2a22098e27 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:59.776270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a29aad-9ac3-40c9-9936-25263e4d6d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:59.779262 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a29aad-9ac3-40c9-9936-25263e4d6d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:59.780709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a29aad-9ac3-40c9-9936-25263e4d6d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:59.781013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a29aad-9ac3-40c9-9936-25263e4d6d50 admin admin] PATH_INFO: `/v3/projects/ff4823e4fae64d1cbae8bfef6ac062fc` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:59.785985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a29aad-9ac3-40c9-9936-25263e4d6d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff4823e4fae64d1cbae8bfef6ac062fc)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:59.788147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a29aad-9ac3-40c9-9936-25263e4d6d50 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:59.984222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-99a29aad-9ac3-40c9-9936-25263e4d6d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff4823e4fae64d1cbae8bfef6ac062fc', 'request_id': 'req-99a29aad-9ac3-40c9-9936-25263e4d6d50'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:00.013524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-99a29aad-9ac3-40c9-9936-25263e4d6d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff4823e4fae64d1cbae8bfef6ac062fc', 'request_id': 'req-99a29aad-9ac3-40c9-9936-25263e4d6d50'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:00.029527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-99a29aad-9ac3-40c9-9936-25263e4d6d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff4823e4fae64d1cbae8bfef6ac062fc', 'request_id': 'req-99a29aad-9ac3-40c9-9936-25263e4d6d50'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:00.084565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-99a29aad-9ac3-40c9-9936-25263e4d6d50 admin admin] The token cache is being invalidate because project ff4823e4fae64d1cbae8bfef6ac062fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:51:00.085030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-99a29aad-9ac3-40c9-9936-25263e4d6d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:00.089541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-99a29aad-9ac3-40c9-9936-25263e4d6d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:00.099784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1155/2313] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:59 2022] DELETE /identity/v3/projects/ff4823e4fae64d1cbae8bfef6ac062fc => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:02.644228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c91665-d54b-41ca-9020-623db6fa8406 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:02.644960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c91665-d54b-41ca-9020-623db6fa8406 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:02.645234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c91665-d54b-41ca-9020-623db6fa8406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:02.671148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-40c91665-d54b-41ca-9020-623db6fa8406 None None] MFA Rules not processed for user `980ab18eb5c646378d738079621a0dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:02.672891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-40c91665-d54b-41ca-9020-623db6fa8406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:02.687368 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:02.723683 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c425f2a-afa2-4680-8107-18985b36d923 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:02.724250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c425f2a-afa2-4680-8107-18985b36d923 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:02.724250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c425f2a-afa2-4680-8107-18985b36d923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:02.760990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-7c425f2a-afa2-4680-8107-18985b36d923 None None] MFA Rules not processed for user `b8818863a81b4238a22035fbb256568e`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:02.762228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7c425f2a-afa2-4680-8107-18985b36d923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:02.781851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1159/2314] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:51:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:02.904901 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1156/2315] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:51:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:51:04.787372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c62a4-52ed-456a-b090-93c4ab7f1b5e None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:04.788243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c62a4-52ed-456a-b090-93c4ab7f1b5e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:04.788675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c62a4-52ed-456a-b090-93c4ab7f1b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:04.826829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-cd2c62a4-52ed-456a-b090-93c4ab7f1b5e None None] MFA Rules not processed for user `893e06c5ea99497fae2847399b7a1a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:04.829103 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-cd2c62a4-52ed-456a-b090-93c4ab7f1b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:05.005717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1160/2316] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:51:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:51:05.012105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c425f2a-afa2-4680-8107-18985b36d923 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:05.013982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7c425f2a-afa2-4680-8107-18985b36d923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:05.121951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c425f2a-afa2-4680-8107-18985b36d923 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:05.129196 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a4383-d777-42ec-af6a-5d39b1f56d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:05.130071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a4383-d777-42ec-af6a-5d39b1f56d9c admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:05.130357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a4383-d777-42ec-af6a-5d39b1f56d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:05.130633 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a4383-d777-42ec-af6a-5d39b1f56d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:05.131338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a4383-d777-42ec-af6a-5d39b1f56d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:05.133375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a4383-d777-42ec-af6a-5d39b1f56d9c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:05.195075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1157/2317] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:51:05 2022] POST /identity/v3/projects => generated 379 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:51:05.198920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c62a4-52ed-456a-b090-93c4ab7f1b5e None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:05.217664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c62a4-52ed-456a-b090-93c4ab7f1b5e None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:05.405522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236dca4-8049-43f7-8576-16f04900534e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:05.407464 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236dca4-8049-43f7-8576-16f04900534e admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:05.409811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236dca4-8049-43f7-8576-16f04900534e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:05.410530 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236dca4-8049-43f7-8576-16f04900534e admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:05.412559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236dca4-8049-43f7-8576-16f04900534e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:05.421385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236dca4-8049-43f7-8576-16f04900534e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:05.480801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1161/2318] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:51:05 2022] POST /identity/v3/users => generated 296 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:51:05.485627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a4383-d777-42ec-af6a-5d39b1f56d9c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:05.521616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a4383-d777-42ec-af6a-5d39b1f56d9c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:05.532817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12d707-6f91-46c3-9c9b-ecdc79403748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:05.533445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d707-6f91-46c3-9c9b-ecdc79403748 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:05.533738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d707-6f91-46c3-9c9b-ecdc79403748 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:05.534035 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d707-6f91-46c3-9c9b-ecdc79403748 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:05.535222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12d707-6f91-46c3-9c9b-ecdc79403748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:05.536863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12d707-6f91-46c3-9c9b-ecdc79403748 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:05.542155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:05.558829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1158/2319] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:05.560086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236dca4-8049-43f7-8576-16f04900534e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:05.574701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236dca4-8049-43f7-8576-16f04900534e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:05.595250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09cbe0-45dd-445c-b490-b649411266ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:05.598919 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09cbe0-45dd-445c-b490-b649411266ad admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:05.599504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09cbe0-45dd-445c-b490-b649411266ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:05.600460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09cbe0-45dd-445c-b490-b649411266ad admin admin] PATH_INFO: `/v3/projects/7ad3ee1d713e4186a2d740ffbb154dc0/users/f030fc172a6748a2865277207656845b/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:05.654622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09cbe0-45dd-445c-b490-b649411266ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad3ee1d713e4186a2d740ffbb154dc0, user_id=f030fc172a6748a2865277207656845b, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:05.658299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09cbe0-45dd-445c-b490-b649411266ad admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:05.700094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1162/2320] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:51:05 2022] PUT /identity/v3/projects/7ad3ee1d713e4186a2d740ffbb154dc0/users/f030fc172a6748a2865277207656845b/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:05.711442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12d707-6f91-46c3-9c9b-ecdc79403748 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:05.743023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12d707-6f91-46c3-9c9b-ecdc79403748 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:05.888935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de860c20-5cee-427b-9a3f-4734139f05cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:05.889754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de860c20-5cee-427b-9a3f-4734139f05cb admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:05.890235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de860c20-5cee-427b-9a3f-4734139f05cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:05.890615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de860c20-5cee-427b-9a3f-4734139f05cb admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:05.891489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de860c20-5cee-427b-9a3f-4734139f05cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:05.895020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de860c20-5cee-427b-9a3f-4734139f05cb admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:05.908921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1159/2321] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:05.913456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09cbe0-45dd-445c-b490-b649411266ad admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:05.933347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09cbe0-45dd-445c-b490-b649411266ad admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:05.938755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19858a3c-4559-4ac0-b7e2-112ce88cb988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:05.939462 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19858a3c-4559-4ac0-b7e2-112ce88cb988 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:05.939744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19858a3c-4559-4ac0-b7e2-112ce88cb988 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:05.940012 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19858a3c-4559-4ac0-b7e2-112ce88cb988 admin admin] PATH_INFO: `/v3/projects/7ad3ee1d713e4186a2d740ffbb154dc0/users/f030fc172a6748a2865277207656845b/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:05.943039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19858a3c-4559-4ac0-b7e2-112ce88cb988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad3ee1d713e4186a2d740ffbb154dc0, user_id=f030fc172a6748a2865277207656845b, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:05.945835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19858a3c-4559-4ac0-b7e2-112ce88cb988 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:05.959630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1163/2322] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:51:05 2022] PUT /identity/v3/projects/7ad3ee1d713e4186a2d740ffbb154dc0/users/f030fc172a6748a2865277207656845b/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:51:13.726037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37df801-2147-4449-a332-2719a7cadfa8 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:13.726586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37df801-2147-4449-a332-2719a7cadfa8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:13.726586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37df801-2147-4449-a332-2719a7cadfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:13.782716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e37df801-2147-4449-a332-2719a7cadfa8 None None] MFA Rules not processed for user `f030fc172a6748a2865277207656845b`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:13.787366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e37df801-2147-4449-a332-2719a7cadfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:13.921388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1160/2323] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:51:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:13.943517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19858a3c-4559-4ac0-b7e2-112ce88cb988 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:13.945482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-19858a3c-4559-4ac0-b7e2-112ce88cb988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:13.955303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:14.143058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19858a3c-4559-4ac0-b7e2-112ce88cb988 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:14.150816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a22631-8181-4828-b48f-2707b6f45a38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:14.151237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a22631-8181-4828-b48f-2707b6f45a38 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:14.151590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a22631-8181-4828-b48f-2707b6f45a38 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:14.151909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a22631-8181-4828-b48f-2707b6f45a38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:14.242534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a22631-8181-4828-b48f-2707b6f45a38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:14.244520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a22631-8181-4828-b48f-2707b6f45a38 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:14.256506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1164/2324] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:51:21.478525 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37df801-2147-4449-a332-2719a7cadfa8 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.480081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e37df801-2147-4449-a332-2719a7cadfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:21.575776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37df801-2147-4449-a332-2719a7cadfa8 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:21.581863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:21.582557 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:21.582842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:21.583149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] PATH_INFO: `/v3/users/79cef444ee0a434c9cd2ef96b7af1536` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:21.584600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=79cef444ee0a434c9cd2ef96b7af1536)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:21.586650 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:21.620750 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:21.668601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79cef444ee0a434c9cd2ef96b7af1536'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:21.699060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79cef444ee0a434c9cd2ef96b7af1536'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:21.700061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79cef444ee0a434c9cd2ef96b7af1536'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:21.700896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79cef444ee0a434c9cd2ef96b7af1536'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:21.720298 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79cef444ee0a434c9cd2ef96b7af1536'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:21.753866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1161/2325] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:51:21 2022] DELETE /identity/v3/users/79cef444ee0a434c9cd2ef96b7af1536 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:21.761254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfee413-dab8-42b9-85c8-9296c4d061b7 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:21.762139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfee413-dab8-42b9-85c8-9296c4d061b7 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:21.762139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfee413-dab8-42b9-85c8-9296c4d061b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:21.809009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-7dfee413-dab8-42b9-85c8-9296c4d061b7 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:21.811329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7dfee413-dab8-42b9-85c8-9296c4d061b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:21.942284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1165/2326] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:21.956363 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.959457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:22.067495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac4fbb-04fb-4cbe-b0ec-ce281769b1ab admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:22.073654 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478be59-239e-4506-b26a-5319dbfa0832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:22.074780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478be59-239e-4506-b26a-5319dbfa0832 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:22.075231 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478be59-239e-4506-b26a-5319dbfa0832 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:22.075653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478be59-239e-4506-b26a-5319dbfa0832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:22.090205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478be59-239e-4506-b26a-5319dbfa0832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:22.091885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478be59-239e-4506-b26a-5319dbfa0832 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:22.103572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1162/2327] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:22.508776 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfee413-dab8-42b9-85c8-9296c4d061b7 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:22.511045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7dfee413-dab8-42b9-85c8-9296c4d061b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:22.533478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfee413-dab8-42b9-85c8-9296c4d061b7 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:22.537125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:22.537797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:22.538074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:22.538344 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] PATH_INFO: `/v3/projects/5a5de530516f49fc97f8c40fc2a64df4` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:22.539646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a5de530516f49fc97f8c40fc2a64df4)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:22.541614 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:22.614941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:22.636266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a5de530516f49fc97f8c40fc2a64df4', 'request_id': 'req-9e057ed5-4f7e-42b8-8e78-482b70434401'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:22.646003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a5de530516f49fc97f8c40fc2a64df4', 'request_id': 'req-9e057ed5-4f7e-42b8-8e78-482b70434401'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:22.654542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a5de530516f49fc97f8c40fc2a64df4', 'request_id': 'req-9e057ed5-4f7e-42b8-8e78-482b70434401'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:22.680362 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] The token cache is being invalidate because project 5a5de530516f49fc97f8c40fc2a64df4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:51:22.680733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:22.682681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:22.685489 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1166/2328] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:51:22 2022] DELETE /identity/v3/projects/5a5de530516f49fc97f8c40fc2a64df4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:51:32.098551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=8,keystone:SELECT=46,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:32.676086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:37.217658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec696bc-7a13-4a8a-b3c6-41a0392218f1 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:37.218198 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec696bc-7a13-4a8a-b3c6-41a0392218f1 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:37.218198 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec696bc-7a13-4a8a-b3c6-41a0392218f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:37.237881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-2ec696bc-7a13-4a8a-b3c6-41a0392218f1 None None] MFA Rules not processed for user `893e06c5ea99497fae2847399b7a1a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:37.240024 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2ec696bc-7a13-4a8a-b3c6-41a0392218f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:37.302685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1163/2329] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:51:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:38.843503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:38.845532 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:38.927003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e057ed5-4f7e-42b8-8e78-482b70434401 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:38.932245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd91745-7d7f-43c5-93a7-9d552d0e5cd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:38.933788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd91745-7d7f-43c5-93a7-9d552d0e5cd8 service placement] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:38.933788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd91745-7d7f-43c5-93a7-9d552d0e5cd8 service placement] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:38.933788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd91745-7d7f-43c5-93a7-9d552d0e5cd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:38.935195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-efd91745-7d7f-43c5-93a7-9d552d0e5cd8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:38.949092 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd91745-7d7f-43c5-93a7-9d552d0e5cd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:38.950695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd91745-7d7f-43c5-93a7-9d552d0e5cd8 service placement] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:38.961206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1167/2330] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:47.294542 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=25 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:48.956555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=32 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:56.318481 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec696bc-7a13-4a8a-b3c6-41a0392218f1 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:56.320135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2ec696bc-7a13-4a8a-b3c6-41a0392218f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:56.417679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec696bc-7a13-4a8a-b3c6-41a0392218f1 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:56.421291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117e23c-1ad0-415c-b4eb-b4686a2fca6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:56.425643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117e23c-1ad0-415c-b4eb-b4686a2fca6c service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:56.425643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117e23c-1ad0-415c-b4eb-b4686a2fca6c service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:56.425643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117e23c-1ad0-415c-b4eb-b4686a2fca6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:56.425643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2117e23c-1ad0-415c-b4eb-b4686a2fca6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:56.506562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117e23c-1ad0-415c-b4eb-b4686a2fca6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:56.508024 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117e23c-1ad0-415c-b4eb-b4686a2fca6c service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:56.518790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1164/2331] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:52:02.971764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea378c90-8f29-4f2d-9a18-eb0e6efa2a9d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:02.972391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea378c90-8f29-4f2d-9a18-eb0e6efa2a9d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:02.972555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea378c90-8f29-4f2d-9a18-eb0e6efa2a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:02.994370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ea378c90-8f29-4f2d-9a18-eb0e6efa2a9d None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:52:02.995683 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ea378c90-8f29-4f2d-9a18-eb0e6efa2a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:03.056454 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1168/2332] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:52:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:52:03.107608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117e23c-1ad0-415c-b4eb-b4686a2fca6c service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:03.109068 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2117e23c-1ad0-415c-b4eb-b4686a2fca6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:03.189309 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117e23c-1ad0-415c-b4eb-b4686a2fca6c service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:03.196985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298671d6-b467-48fc-bbc7-cf86e57d36c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:03.197639 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298671d6-b467-48fc-bbc7-cf86e57d36c9 service swift] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:03.197925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298671d6-b467-48fc-bbc7-cf86e57d36c9 service swift] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:03.198266 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298671d6-b467-48fc-bbc7-cf86e57d36c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:03.216595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298671d6-b467-48fc-bbc7-cf86e57d36c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:03.219749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298671d6-b467-48fc-bbc7-cf86e57d36c9 service swift] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:03.233307 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1165/2333] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:52:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:52:13.048806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=25 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:13.230100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=82 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:19.972710 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a2b5f-09a7-48ad-9f83-80f44a22f8bb None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:19.973301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a2b5f-09a7-48ad-9f83-80f44a22f8bb None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:19.973301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a2b5f-09a7-48ad-9f83-80f44a22f8bb None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:19.975661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1169/2334] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 09:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:52:19.981388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0721fb0-ffc4-4a5b-a40b-66bf1822ecdb None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:19.981924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0721fb0-ffc4-4a5b-a40b-66bf1822ecdb None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:19.982207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0721fb0-ffc4-4a5b-a40b-66bf1822ecdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:20.027545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e0721fb0-ffc4-4a5b-a40b-66bf1822ecdb None None] MFA Rules not processed for user `ef72bb95992c4a21ae2dbb2a08c51421`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:52:20.029202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e0721fb0-ffc4-4a5b-a40b-66bf1822ecdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:20.121505 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1166/2335] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 09:52:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:52:20.135091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a2b5f-09a7-48ad-9f83-80f44a22f8bb None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:20.137315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-151a2b5f-09a7-48ad-9f83-80f44a22f8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:20.209551 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a2b5f-09a7-48ad-9f83-80f44a22f8bb None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:20.214016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0553b-f4a3-44db-9cbe-7bffe1d1a3dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06fad9466df14827983dac2b4b844f3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:20.214838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0553b-f4a3-44db-9cbe-7bffe1d1a3dc service glance] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:20.215305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0553b-f4a3-44db-9cbe-7bffe1d1a3dc service glance] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:20.215713 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0553b-f4a3-44db-9cbe-7bffe1d1a3dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:20.227605 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0553b-f4a3-44db-9cbe-7bffe1d1a3dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:20.229480 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0553b-f4a3-44db-9cbe-7bffe1d1a3dc service glance] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:20.243073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1170/2336] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:52:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:52:20.356273 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f46854-cf27-47ff-afd2-69d5273bab48 None None] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:20.356712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f46854-cf27-47ff-afd2-69d5273bab48 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:20.357198 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f46854-cf27-47ff-afd2-69d5273bab48 None None] PATH_INFO: `/` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:20.359529 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1167/2337] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 09:52:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:52:20.365753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328533e-5746-4f4a-96af-60b64bf168fb None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:20.366376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328533e-5746-4f4a-96af-60b64bf168fb None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:20.366879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328533e-5746-4f4a-96af-60b64bf168fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:20.411355 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4328533e-5746-4f4a-96af-60b64bf168fb None None] MFA Rules not processed for user `6ff1177119cb48e0a88b0f11a6b1b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:52:20.412648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4328533e-5746-4f4a-96af-60b64bf168fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:20.424219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1171/2338] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 09:52:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:52:20.437764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f46854-cf27-47ff-afd2-69d5273bab48 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:20.453833 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f46854-cf27-47ff-afd2-69d5273bab48 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:20.460659 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dba4b9-499a-427e-8c21-802e9dfa10d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:20.461586 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dba4b9-499a-427e-8c21-802e9dfa10d9 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:20.461963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dba4b9-499a-427e-8c21-802e9dfa10d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:20.462639 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dba4b9-499a-427e-8c21-802e9dfa10d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:20.473285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dba4b9-499a-427e-8c21-802e9dfa10d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:20.476513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dba4b9-499a-427e-8c21-802e9dfa10d9 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:20.486887 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1168/2339] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:52:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:52:20.585768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7713c4-a215-4f64-a6e0-f035403e46e9 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:20.586417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7713c4-a215-4f64-a6e0-f035403e46e9 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:20.586766 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7713c4-a215-4f64-a6e0-f035403e46e9 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:20.588378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1172/2340] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 09:52:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:52:20.596255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d51a6-a5f3-4cf5-a8c3-0c103ac06fb9 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:20.596707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d51a6-a5f3-4cf5-a8c3-0c103ac06fb9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:20.597651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d51a6-a5f3-4cf5-a8c3-0c103ac06fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:20.629558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-1e5d51a6-a5f3-4cf5-a8c3-0c103ac06fb9 None None] MFA Rules not processed for user `ef72bb95992c4a21ae2dbb2a08c51421`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:52:20.630916 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1e5d51a6-a5f3-4cf5-a8c3-0c103ac06fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:20.641386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1169/2341] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 09:52:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:52:20.653765 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7713c4-a215-4f64-a6e0-f035403e46e9 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:20.670898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7713c4-a215-4f64-a6e0-f035403e46e9 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:20.675326 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb71fa-9f8b-475b-a7fc-b06f9b14bcb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:20.676524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb71fa-9f8b-475b-a7fc-b06f9b14bcb7 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:20.676801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb71fa-9f8b-475b-a7fc-b06f9b14bcb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:20.677207 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb71fa-9f8b-475b-a7fc-b06f9b14bcb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:20.691631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb71fa-9f8b-475b-a7fc-b06f9b14bcb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:20.692686 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb71fa-9f8b-475b-a7fc-b06f9b14bcb7 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:20.707166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1173/2342] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:52:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:52:28.834881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83c8bd-e5e6-41e3-b2da-c0fa2b0a5dad None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:28.835669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83c8bd-e5e6-41e3-b2da-c0fa2b0a5dad None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:28.835669 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83c8bd-e5e6-41e3-b2da-c0fa2b0a5dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:28.857080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ae83c8bd-e5e6-41e3-b2da-c0fa2b0a5dad None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:52:28.858354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ae83c8bd-e5e6-41e3-b2da-c0fa2b0a5dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:28.868123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1170/2343] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:52:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:52:28.881107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb71fa-9f8b-475b-a7fc-b06f9b14bcb7 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:28.892452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb71fa-9f8b-475b-a7fc-b06f9b14bcb7 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:28.898165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70697cf5-9b6d-4094-b01c-29eb786e2f34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:28.898791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70697cf5-9b6d-4094-b01c-29eb786e2f34 service swift] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:28.899082 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70697cf5-9b6d-4094-b01c-29eb786e2f34 service swift] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:28.899405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70697cf5-9b6d-4094-b01c-29eb786e2f34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:28.908200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70697cf5-9b6d-4094-b01c-29eb786e2f34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:28.910304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70697cf5-9b6d-4094-b01c-29eb786e2f34 service swift] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:28.918495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1174/2344] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:52:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:52:38.853352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=43 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:38.915204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=41 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:47.100071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2903894-2ac0-4351-8029-036dd7047fbe None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:47.100868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2903894-2ac0-4351-8029-036dd7047fbe None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:47.100868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2903894-2ac0-4351-8029-036dd7047fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:47.122268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-d2903894-2ac0-4351-8029-036dd7047fbe None None] MFA Rules not processed for user `980ab18eb5c646378d738079621a0dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:52:47.123642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d2903894-2ac0-4351-8029-036dd7047fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:47.208050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1171/2345] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:52:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:52:53.647390 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4097ef-92f3-4744-b48a-0ed063eb303f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:53.647971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4097ef-92f3-4744-b48a-0ed063eb303f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:53.647971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4097ef-92f3-4744-b48a-0ed063eb303f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:53.667392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ff4097ef-92f3-4744-b48a-0ed063eb303f None None] MFA Rules not processed for user `b8818863a81b4238a22035fbb256568e`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:52:53.668612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ff4097ef-92f3-4744-b48a-0ed063eb303f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:53.709803 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1863fe-49ab-4790-8d68-5c4088b2c277 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:53.710661 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1863fe-49ab-4790-8d68-5c4088b2c277 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:53.711383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1863fe-49ab-4790-8d68-5c4088b2c277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:53.734980 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1175/2346] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:52:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:52:53.736572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ed1863fe-49ab-4790-8d68-5c4088b2c277 None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:52:53.739799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ed1863fe-49ab-4790-8d68-5c4088b2c277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:53.747831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1172/2347] 10.222.0.36 () {58 vars in 967 bytes} [Wed Aug 31 09:52:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:52:53.763408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4097ef-92f3-4744-b48a-0ed063eb303f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:53.777181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4097ef-92f3-4744-b48a-0ed063eb303f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:53.782201 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd145462-36b8-4836-8318-22defd507d6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:53.782801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd145462-36b8-4836-8318-22defd507d6b service swift] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:53.784303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd145462-36b8-4836-8318-22defd507d6b service swift] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:53.784303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd145462-36b8-4836-8318-22defd507d6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:53.793896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd145462-36b8-4836-8318-22defd507d6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:53.795538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd145462-36b8-4836-8318-22defd507d6b service swift] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:53.804346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1176/2348] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:52:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:53:03.728352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=28 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:03.801389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=31 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:22.580659 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1863fe-49ab-4790-8d68-5c4088b2c277 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:22.582149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ed1863fe-49ab-4790-8d68-5c4088b2c277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:22.593802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1863fe-49ab-4790-8d68-5c4088b2c277 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:22.597297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35adc924-21ac-4d77-8e94-bfb5f19b4e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:22.597896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adc924-21ac-4d77-8e94-bfb5f19b4e7f service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:22.598178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adc924-21ac-4d77-8e94-bfb5f19b4e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:22.598447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adc924-21ac-4d77-8e94-bfb5f19b4e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:22.600178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-35adc924-21ac-4d77-8e94-bfb5f19b4e7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:22.613977 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35adc924-21ac-4d77-8e94-bfb5f19b4e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:22.615437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35adc924-21ac-4d77-8e94-bfb5f19b4e7f service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:22.627391 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1173/2349] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:23.812045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd145462-36b8-4836-8318-22defd507d6b service swift] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:23.821732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd145462-36b8-4836-8318-22defd507d6b service swift] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:23.825373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380fc69-d1a1-4425-a5ae-d4b81a39095a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:23.825966 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380fc69-d1a1-4425-a5ae-d4b81a39095a service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:23.826245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380fc69-d1a1-4425-a5ae-d4b81a39095a service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:23.826516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380fc69-d1a1-4425-a5ae-d4b81a39095a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:23.828295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1380fc69-d1a1-4425-a5ae-d4b81a39095a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:23.842142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380fc69-d1a1-4425-a5ae-d4b81a39095a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:23.843604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380fc69-d1a1-4425-a5ae-d4b81a39095a service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:23.855850 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1177/2350] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:29.420374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a5b06-4844-4ca0-bc68-2b0b90fda0cc None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:29.420989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a5b06-4844-4ca0-bc68-2b0b90fda0cc None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:29.421161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a5b06-4844-4ca0-bc68-2b0b90fda0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:29.451621 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-a55a5b06-4844-4ca0-bc68-2b0b90fda0cc None None] MFA Rules not processed for user `b6ca4276e1554d4184451313b2e09a88`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:29.452867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a55a5b06-4844-4ca0-bc68-2b0b90fda0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:29.518772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1174/2351] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:53:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:33.849920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=6 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:38.652218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380fc69-d1a1-4425-a5ae-d4b81a39095a service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:38.653689 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1380fc69-d1a1-4425-a5ae-d4b81a39095a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:38.669846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380fc69-d1a1-4425-a5ae-d4b81a39095a service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:38.675531 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c9182-64ad-4a81-9ce1-28ceedd21c53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:38.676132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c9182-64ad-4a81-9ce1-28ceedd21c53 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:38.676408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c9182-64ad-4a81-9ce1-28ceedd21c53 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:38.676672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c9182-64ad-4a81-9ce1-28ceedd21c53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:38.679613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e97c9182-64ad-4a81-9ce1-28ceedd21c53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:38.690412 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c9182-64ad-4a81-9ce1-28ceedd21c53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:38.692096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c9182-64ad-4a81-9ce1-28ceedd21c53 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:38.702459 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1178/2352] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:39.033278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a5b06-4844-4ca0-bc68-2b0b90fda0cc None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:39.034672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a55a5b06-4844-4ca0-bc68-2b0b90fda0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:39.110062 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a5b06-4844-4ca0-bc68-2b0b90fda0cc None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:39.113899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8122fdf-4e62-4b85-b321-b224b55976db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:39.114515 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8122fdf-4e62-4b85-b321-b224b55976db service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:39.114796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8122fdf-4e62-4b85-b321-b224b55976db service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:39.115064 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8122fdf-4e62-4b85-b321-b224b55976db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:39.118590 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b8122fdf-4e62-4b85-b321-b224b55976db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:39.131448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8122fdf-4e62-4b85-b321-b224b55976db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:39.132810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8122fdf-4e62-4b85-b321-b224b55976db service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:39.146341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1175/2353] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:45.082261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fdb6c-923d-4dde-b45f-97c7713f648f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:45.082742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fdb6c-923d-4dde-b45f-97c7713f648f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:45.082844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fdb6c-923d-4dde-b45f-97c7713f648f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:45.104323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-1f6fdb6c-923d-4dde-b45f-97c7713f648f None None] MFA Rules not processed for user `18eecdfed8cc44f685095b6a70a4e69d`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:45.105613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1f6fdb6c-923d-4dde-b45f-97c7713f648f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:45.160468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1179/2354] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:53:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:45.475359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8122fdf-4e62-4b85-b321-b224b55976db service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:45.479589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=58 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:45.485520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8122fdf-4e62-4b85-b321-b224b55976db service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:45.489085 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671086e-e105-4f14-83f4-28a2ae857824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:45.489673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671086e-e105-4f14-83f4-28a2ae857824 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:45.489957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671086e-e105-4f14-83f4-28a2ae857824 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:45.490221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671086e-e105-4f14-83f4-28a2ae857824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:45.492240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a671086e-e105-4f14-83f4-28a2ae857824 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:45.501985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671086e-e105-4f14-83f4-28a2ae857824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:45.503443 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671086e-e105-4f14-83f4-28a2ae857824 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:45.512148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1176/2355] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:45.844435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668c52e-8bd0-4be2-8ea4-5b9e20112b90 None None] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:45.844717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668c52e-8bd0-4be2-8ea4-5b9e20112b90 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:45.844996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668c52e-8bd0-4be2-8ea4-5b9e20112b90 None None] PATH_INFO: `/` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:45.846330 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1180/2356] 10.222.0.36 () {54 vars in 961 bytes} [Wed Aug 31 09:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:53:45.852288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1efd20-418c-41f8-853b-1d69d6d29e9b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:45.852741 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1efd20-418c-41f8-853b-1d69d6d29e9b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:45.853233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1efd20-418c-41f8-853b-1d69d6d29e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:45.874567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-df1efd20-418c-41f8-853b-1d69d6d29e9b None None] MFA Rules not processed for user `560df3eb05f14b6489158ab053f282ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:45.876030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-df1efd20-418c-41f8-853b-1d69d6d29e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:45.884884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1177/2357] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Aug 31 09:53:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:45.892861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668c52e-8bd0-4be2-8ea4-5b9e20112b90 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:45.906767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668c52e-8bd0-4be2-8ea4-5b9e20112b90 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:45.911723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649d95d-b6a4-4203-8b12-789d37f2de97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:45.913037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649d95d-b6a4-4203-8b12-789d37f2de97 service placement] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:45.913239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649d95d-b6a4-4203-8b12-789d37f2de97 service placement] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:45.913513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649d95d-b6a4-4203-8b12-789d37f2de97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:45.915291 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d649d95d-b6a4-4203-8b12-789d37f2de97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:45.927802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649d95d-b6a4-4203-8b12-789d37f2de97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:45.929407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649d95d-b6a4-4203-8b12-789d37f2de97 service placement] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:45.939417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1181/2358] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:48.066183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1efd20-418c-41f8-853b-1d69d6d29e9b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:48.067724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-df1efd20-418c-41f8-853b-1d69d6d29e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:48.090442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1efd20-418c-41f8-853b-1d69d6d29e9b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:48.097236 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387f592-a628-4585-a79f-3f8a57e07e06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:48.098536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387f592-a628-4585-a79f-3f8a57e07e06 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:48.099078 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387f592-a628-4585-a79f-3f8a57e07e06 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:48.099628 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387f592-a628-4585-a79f-3f8a57e07e06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:48.103433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d387f592-a628-4585-a79f-3f8a57e07e06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:48.117915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387f592-a628-4585-a79f-3f8a57e07e06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:48.119917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387f592-a628-4585-a79f-3f8a57e07e06 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:48.134166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1178/2359] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:55.935099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=37 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:56.222447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649d95d-b6a4-4203-8b12-789d37f2de97 service placement] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:56.232447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649d95d-b6a4-4203-8b12-789d37f2de97 service placement] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:56.236174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdbefb-959d-4f16-a266-7d5387277e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:56.236706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdbefb-959d-4f16-a266-7d5387277e72 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:56.236980 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdbefb-959d-4f16-a266-7d5387277e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:56.237254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdbefb-959d-4f16-a266-7d5387277e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:56.238993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3ccdbefb-959d-4f16-a266-7d5387277e72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:56.249386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdbefb-959d-4f16-a266-7d5387277e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:56.250811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdbefb-959d-4f16-a266-7d5387277e72 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:56.259078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1182/2360] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:53:58.127199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=14 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:00.491561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387f592-a628-4585-a79f-3f8a57e07e06 service nova] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:00.508287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387f592-a628-4585-a79f-3f8a57e07e06 service nova] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:00.515614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f08ff-465f-4930-8cb3-d5f48f7deaa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:00.516552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f08ff-465f-4930-8cb3-d5f48f7deaa8 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:00.516991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f08ff-465f-4930-8cb3-d5f48f7deaa8 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:00.517420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f08ff-465f-4930-8cb3-d5f48f7deaa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:00.519567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a00f08ff-465f-4930-8cb3-d5f48f7deaa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:00.546274 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f08ff-465f-4930-8cb3-d5f48f7deaa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:00.547913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f08ff-465f-4930-8cb3-d5f48f7deaa8 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:00.559847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1179/2361] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:06.255457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=6 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:08.403766 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdbefb-959d-4f16-a266-7d5387277e72 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:08.412385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdbefb-959d-4f16-a266-7d5387277e72 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:08.416668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f620f-6c0a-42c6-b91f-1633dfcfb0b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:08.417271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f620f-6c0a-42c6-b91f-1633dfcfb0b5 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:08.417540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f620f-6c0a-42c6-b91f-1633dfcfb0b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:08.417803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f620f-6c0a-42c6-b91f-1633dfcfb0b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:08.419575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-390f620f-6c0a-42c6-b91f-1633dfcfb0b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:08.487922 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f620f-6c0a-42c6-b91f-1633dfcfb0b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:08.489366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f620f-6c0a-42c6-b91f-1633dfcfb0b5 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:08.499865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1183/2362] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:10.555940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=6 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:11.672246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf4e71-5ea1-4cf8-8a79-d3eded1721cb None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:11.672908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf4e71-5ea1-4cf8-8a79-d3eded1721cb None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:11.672908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf4e71-5ea1-4cf8-8a79-d3eded1721cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:11.701728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-1cdf4e71-5ea1-4cf8-8a79-d3eded1721cb None None] MFA Rules not processed for user `893e06c5ea99497fae2847399b7a1a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:11.703773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1cdf4e71-5ea1-4cf8-8a79-d3eded1721cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:11.717982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1180/2363] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:13.017607 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f620f-6c0a-42c6-b91f-1633dfcfb0b5 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:13.029109 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f620f-6c0a-42c6-b91f-1633dfcfb0b5 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:13.034087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f95cb-39c0-499c-8a51-6222b73e5be3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:13.034689 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f95cb-39c0-499c-8a51-6222b73e5be3 service placement] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:13.034964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f95cb-39c0-499c-8a51-6222b73e5be3 service placement] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:13.035274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f95cb-39c0-499c-8a51-6222b73e5be3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:13.036995 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0c0f95cb-39c0-499c-8a51-6222b73e5be3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:13.049205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f95cb-39c0-499c-8a51-6222b73e5be3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:13.050781 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f95cb-39c0-499c-8a51-6222b73e5be3 service placement] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:13.065578 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1184/2364] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:13.136301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf4e71-5ea1-4cf8-8a79-d3eded1721cb None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:13.138487 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1cdf4e71-5ea1-4cf8-8a79-d3eded1721cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:13.164044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf4e71-5ea1-4cf8-8a79-d3eded1721cb None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:13.170713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afcf82-a85a-44e0-af89-e29faf67d96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:13.171678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afcf82-a85a-44e0-af89-e29faf67d96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:13.172168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afcf82-a85a-44e0-af89-e29faf67d96d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:13.172632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afcf82-a85a-44e0-af89-e29faf67d96d admin admin] PATH_INFO: `/v3/users/b8818863a81b4238a22035fbb256568e` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:13.173987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45afcf82-a85a-44e0-af89-e29faf67d96d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8818863a81b4238a22035fbb256568e)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:13.176529 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45afcf82-a85a-44e0-af89-e29faf67d96d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:13.249512 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-45afcf82-a85a-44e0-af89-e29faf67d96d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8818863a81b4238a22035fbb256568e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:13.266567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-45afcf82-a85a-44e0-af89-e29faf67d96d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8818863a81b4238a22035fbb256568e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:13.268704 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-45afcf82-a85a-44e0-af89-e29faf67d96d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8818863a81b4238a22035fbb256568e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:13.269289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-45afcf82-a85a-44e0-af89-e29faf67d96d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8818863a81b4238a22035fbb256568e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:13.281482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-45afcf82-a85a-44e0-af89-e29faf67d96d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8818863a81b4238a22035fbb256568e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:13.311165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1181/2365] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:54:13 2022] DELETE /identity/v3/users/b8818863a81b4238a22035fbb256568e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:13.545108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f95cb-39c0-499c-8a51-6222b73e5be3 service placement] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:13.558556 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f95cb-39c0-499c-8a51-6222b73e5be3 service placement] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:13.647304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196d5d7-11ab-45e1-99e1-e7e08831ed64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:13.648775 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196d5d7-11ab-45e1-99e1-e7e08831ed64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:13.649297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196d5d7-11ab-45e1-99e1-e7e08831ed64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:13.649797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196d5d7-11ab-45e1-99e1-e7e08831ed64 admin admin] PATH_INFO: `/v3/projects/bfef7bc61a644c12a42cc17b1aed1b31` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:13.651210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196d5d7-11ab-45e1-99e1-e7e08831ed64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfef7bc61a644c12a42cc17b1aed1b31)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:13.653328 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196d5d7-11ab-45e1-99e1-e7e08831ed64 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:13.722345 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4196d5d7-11ab-45e1-99e1-e7e08831ed64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfef7bc61a644c12a42cc17b1aed1b31', 'request_id': 'req-4196d5d7-11ab-45e1-99e1-e7e08831ed64'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:13.732117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4196d5d7-11ab-45e1-99e1-e7e08831ed64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfef7bc61a644c12a42cc17b1aed1b31', 'request_id': 'req-4196d5d7-11ab-45e1-99e1-e7e08831ed64'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:13.739022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4196d5d7-11ab-45e1-99e1-e7e08831ed64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfef7bc61a644c12a42cc17b1aed1b31', 'request_id': 'req-4196d5d7-11ab-45e1-99e1-e7e08831ed64'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:13.756450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4196d5d7-11ab-45e1-99e1-e7e08831ed64 admin admin] The token cache is being invalidate because project bfef7bc61a644c12a42cc17b1aed1b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:54:13.757219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4196d5d7-11ab-45e1-99e1-e7e08831ed64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:13.757901 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4196d5d7-11ab-45e1-99e1-e7e08831ed64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:13.764295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1185/2366] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:54:13 2022] DELETE /identity/v3/projects/bfef7bc61a644c12a42cc17b1aed1b31 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:13.765376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a498c8-5bc8-4b20-b87c-e74131c447d9 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:13.765671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a498c8-5bc8-4b20-b87c-e74131c447d9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:13.765950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a498c8-5bc8-4b20-b87c-e74131c447d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:13.789799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-17a498c8-5bc8-4b20-b87c-e74131c447d9 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:13.791555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-17a498c8-5bc8-4b20-b87c-e74131c447d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:13.872939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1182/2367] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:54:13.880219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb45a5-2706-4491-824b-0b12ea4cf196 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:13.880734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb45a5-2706-4491-824b-0b12ea4cf196 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:13.881185 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb45a5-2706-4491-824b-0b12ea4cf196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:13.904205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-6beb45a5-2706-4491-824b-0b12ea4cf196 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:13.905433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6beb45a5-2706-4491-824b-0b12ea4cf196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:13.918985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1186/2368] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:13.920002 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a498c8-5bc8-4b20-b87c-e74131c447d9 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:13.932324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a498c8-5bc8-4b20-b87c-e74131c447d9 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:13.937184 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7b7f4-e3ca-4b6d-89e2-8a0fcb8cec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:13.937817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7b7f4-e3ca-4b6d-89e2-8a0fcb8cec7d admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:13.938100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7b7f4-e3ca-4b6d-89e2-8a0fcb8cec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:13.938380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7b7f4-e3ca-4b6d-89e2-8a0fcb8cec7d admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:13.939041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7b7f4-e3ca-4b6d-89e2-8a0fcb8cec7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:54:13.939382 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7b7f4-e3ca-4b6d-89e2-8a0fcb8cec7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:13.941016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7b7f4-e3ca-4b6d-89e2-8a0fcb8cec7d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:13.954261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1183/2369] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:54:13.958989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb45a5-2706-4491-824b-0b12ea4cf196 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:13.966342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:13.971715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb45a5-2706-4491-824b-0b12ea4cf196 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:13.981288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b7969-f08d-4121-ba56-01668e36a447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:13.982659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b7969-f08d-4121-ba56-01668e36a447 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:13.984680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b7969-f08d-4121-ba56-01668e36a447 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:13.984680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b7969-f08d-4121-ba56-01668e36a447 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:13.985213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b7969-f08d-4121-ba56-01668e36a447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:13.987783 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b7969-f08d-4121-ba56-01668e36a447 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:14.034927 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1187/2370] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:54:13 2022] POST /identity/v3/projects => generated 365 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:54:14.039371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7b7f4-e3ca-4b6d-89e2-8a0fcb8cec7d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:14.049560 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7b7f4-e3ca-4b6d-89e2-8a0fcb8cec7d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:14.115059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:14.133074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955514b-fcd7-4e03-b656-d45236d0baba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:14.133706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955514b-fcd7-4e03-b656-d45236d0baba admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:14.134015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955514b-fcd7-4e03-b656-d45236d0baba admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:14.134292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955514b-fcd7-4e03-b656-d45236d0baba admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:14.134932 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955514b-fcd7-4e03-b656-d45236d0baba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:14.136907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955514b-fcd7-4e03-b656-d45236d0baba admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:14.141589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326c2b5-1b0d-499b-bb06-a6689d5d26c3 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:14.141895 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326c2b5-1b0d-499b-bb06-a6689d5d26c3 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:14.142142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326c2b5-1b0d-499b-bb06-a6689d5d26c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:14.172270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1184/2371] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:54:14 2022] POST /identity/v3/users => generated 283 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:54:14.173321 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955514b-fcd7-4e03-b656-d45236d0baba admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:14.179307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-9326c2b5-1b0d-499b-bb06-a6689d5d26c3 None None] MFA Rules not processed for user `980ab18eb5c646378d738079621a0dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:14.181314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9326c2b5-1b0d-499b-bb06-a6689d5d26c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:14.185645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955514b-fcd7-4e03-b656-d45236d0baba admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:14.191790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4d0ff-8674-4c68-9421-6f92428d5c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:14.192652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4d0ff-8674-4c68-9421-6f92428d5c25 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:14.193032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4d0ff-8674-4c68-9421-6f92428d5c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:14.193823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4d0ff-8674-4c68-9421-6f92428d5c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:14.194600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4d0ff-8674-4c68-9421-6f92428d5c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:14.196778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4d0ff-8674-4c68-9421-6f92428d5c25 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:14.209021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1185/2372] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:14.213734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4d0ff-8674-4c68-9421-6f92428d5c25 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:14.225421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4d0ff-8674-4c68-9421-6f92428d5c25 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:14.231989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33868a01-9ce6-460b-9590-5c11a369ba0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:14.232720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33868a01-9ce6-460b-9590-5c11a369ba0e admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:14.233004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33868a01-9ce6-460b-9590-5c11a369ba0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:14.233273 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33868a01-9ce6-460b-9590-5c11a369ba0e admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:14.234092 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33868a01-9ce6-460b-9590-5c11a369ba0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:14.236076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33868a01-9ce6-460b-9590-5c11a369ba0e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:14.246933 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1188/2373] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:14.251292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1186/2374] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:54:14.252647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326c2b5-1b0d-499b-bb06-a6689d5d26c3 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:14.263868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326c2b5-1b0d-499b-bb06-a6689d5d26c3 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:14.268908 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52882014-bf7e-43dd-bff5-5d7d4fcc50cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:14.269679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52882014-bf7e-43dd-bff5-5d7d4fcc50cc admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:14.269958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52882014-bf7e-43dd-bff5-5d7d4fcc50cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:14.270239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52882014-bf7e-43dd-bff5-5d7d4fcc50cc admin admin] PATH_INFO: `/v3/projects/1f256e4acd3f4ff9b831685d88a2a179/users/26ffbd84a77645be87dd5f8fa0086116/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:14.293066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52882014-bf7e-43dd-bff5-5d7d4fcc50cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f256e4acd3f4ff9b831685d88a2a179, user_id=26ffbd84a77645be87dd5f8fa0086116, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:14.295898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52882014-bf7e-43dd-bff5-5d7d4fcc50cc admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:14.312180 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1189/2375] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:14 2022] PUT /identity/v3/projects/1f256e4acd3f4ff9b831685d88a2a179/users/26ffbd84a77645be87dd5f8fa0086116/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:14.325128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a569f87-f1b4-41ca-a32a-d152f209ca93 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:14.325522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a569f87-f1b4-41ca-a32a-d152f209ca93 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:14.326009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a569f87-f1b4-41ca-a32a-d152f209ca93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:14.358182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0a569f87-f1b4-41ca-a32a-d152f209ca93 None None] MFA Rules not processed for user `26ffbd84a77645be87dd5f8fa0086116`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:14.359672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0a569f87-f1b4-41ca-a32a-d152f209ca93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:14.430168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1187/2376] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:14.434740 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52882014-bf7e-43dd-bff5-5d7d4fcc50cc admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:14.500576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52882014-bf7e-43dd-bff5-5d7d4fcc50cc admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:14.505716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb065df2-82d6-44fa-a9f3-114cf96039db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:14.506338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb065df2-82d6-44fa-a9f3-114cf96039db admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:14.506616 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb065df2-82d6-44fa-a9f3-114cf96039db admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:14.507088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb065df2-82d6-44fa-a9f3-114cf96039db admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:14.507968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb065df2-82d6-44fa-a9f3-114cf96039db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:14.509891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb065df2-82d6-44fa-a9f3-114cf96039db admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:14.541341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1190/2377] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:54:14 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:54:14.545451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a569f87-f1b4-41ca-a32a-d152f209ca93 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:14.557280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a569f87-f1b4-41ca-a32a-d152f209ca93 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:14.653214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b4f5e-4044-45e6-8eac-04f1890afa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:14.655193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b4f5e-4044-45e6-8eac-04f1890afa59 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:14.657170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b4f5e-4044-45e6-8eac-04f1890afa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:14.657565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b4f5e-4044-45e6-8eac-04f1890afa59 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:14.658521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b4f5e-4044-45e6-8eac-04f1890afa59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:14.661238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b4f5e-4044-45e6-8eac-04f1890afa59 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:14.688280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1188/2378] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:54:14 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:54:14.692259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb065df2-82d6-44fa-a9f3-114cf96039db admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:14.711751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb065df2-82d6-44fa-a9f3-114cf96039db admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:14.717035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4758a-2554-4f09-b020-d85c24e24641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:14.717985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4758a-2554-4f09-b020-d85c24e24641 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:14.718571 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4758a-2554-4f09-b020-d85c24e24641 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:14.718719 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4758a-2554-4f09-b020-d85c24e24641 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:14.719413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4758a-2554-4f09-b020-d85c24e24641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:14.721285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4758a-2554-4f09-b020-d85c24e24641 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:14.734639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1191/2379] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:54:14.739033 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b4f5e-4044-45e6-8eac-04f1890afa59 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:14.750768 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b4f5e-4044-45e6-8eac-04f1890afa59 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:14.755520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ba4ee-d415-43c8-8ae5-ee86e8c10fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:14.756175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ba4ee-d415-43c8-8ae5-ee86e8c10fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:14.756449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ba4ee-d415-43c8-8ae5-ee86e8c10fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:14.756749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ba4ee-d415-43c8-8ae5-ee86e8c10fd0 admin admin] PATH_INFO: `/v3/projects/31e2015c6d3e4ae8ae44c87c5a984eaa/users/9340f4cc8bbf49d7b240d26ef8f37dca/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:14.781201 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ba4ee-d415-43c8-8ae5-ee86e8c10fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e2015c6d3e4ae8ae44c87c5a984eaa, user_id=9340f4cc8bbf49d7b240d26ef8f37dca, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:14.784071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ba4ee-d415-43c8-8ae5-ee86e8c10fd0 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:14.799292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1189/2380] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:14 2022] PUT /identity/v3/projects/31e2015c6d3e4ae8ae44c87c5a984eaa/users/9340f4cc8bbf49d7b240d26ef8f37dca/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:14.803700 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4758a-2554-4f09-b020-d85c24e24641 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:14.815753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4758a-2554-4f09-b020-d85c24e24641 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:14.881501 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:14.926397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811798e6-459f-4e70-86ba-175edb16cd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:14.927114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811798e6-459f-4e70-86ba-175edb16cd38 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:14.927509 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811798e6-459f-4e70-86ba-175edb16cd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:14.928472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811798e6-459f-4e70-86ba-175edb16cd38 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:14.929861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811798e6-459f-4e70-86ba-175edb16cd38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:14.931809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811798e6-459f-4e70-86ba-175edb16cd38 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:14.946386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1192/2381] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:14.950344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ba4ee-d415-43c8-8ae5-ee86e8c10fd0 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:14.963236 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ba4ee-d415-43c8-8ae5-ee86e8c10fd0 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:14.970718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c40851-535b-44ed-8b2f-42838b0ce6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:14.972341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c40851-535b-44ed-8b2f-42838b0ce6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:14.972839 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c40851-535b-44ed-8b2f-42838b0ce6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:14.973329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c40851-535b-44ed-8b2f-42838b0ce6fa admin admin] PATH_INFO: `/v3/projects/31e2015c6d3e4ae8ae44c87c5a984eaa/users/9340f4cc8bbf49d7b240d26ef8f37dca/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:14.977271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c40851-535b-44ed-8b2f-42838b0ce6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e2015c6d3e4ae8ae44c87c5a984eaa, user_id=9340f4cc8bbf49d7b240d26ef8f37dca, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:14.980256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c40851-535b-44ed-8b2f-42838b0ce6fa admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:14.992243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1190/2382] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:14 2022] PUT /identity/v3/projects/31e2015c6d3e4ae8ae44c87c5a984eaa/users/9340f4cc8bbf49d7b240d26ef8f37dca/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:15.002835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98081872-7cab-4d28-a006-addf5a2359ce None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:15.003141 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98081872-7cab-4d28-a006-addf5a2359ce None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:15.003596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98081872-7cab-4d28-a006-addf5a2359ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:15.083400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-98081872-7cab-4d28-a006-addf5a2359ce None None] MFA Rules not processed for user `9340f4cc8bbf49d7b240d26ef8f37dca`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:15.085960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-98081872-7cab-4d28-a006-addf5a2359ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:15.182902 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1193/2383] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:15.190897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c40851-535b-44ed-8b2f-42838b0ce6fa admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:15.192947 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-12c40851-535b-44ed-8b2f-42838b0ce6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:15.199935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:15.275789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c40851-535b-44ed-8b2f-42838b0ce6fa admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:15.281145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef98ae-e81d-480e-9e9c-e52f32dc2bd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:15.282044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef98ae-e81d-480e-9e9c-e52f32dc2bd6 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:15.282465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef98ae-e81d-480e-9e9c-e52f32dc2bd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:15.282734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef98ae-e81d-480e-9e9c-e52f32dc2bd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:15.297690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef98ae-e81d-480e-9e9c-e52f32dc2bd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:15.299162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef98ae-e81d-480e-9e9c-e52f32dc2bd6 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:15.311905 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1191/2384] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:15 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:15.847799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98081872-7cab-4d28-a006-addf5a2359ce None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:15.948389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98081872-7cab-4d28-a006-addf5a2359ce None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:15.953200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:15.953931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:15.954213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:15.954483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] PATH_INFO: `/v3/users/26ffbd84a77645be87dd5f8fa0086116` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:15.955879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ffbd84a77645be87dd5f8fa0086116)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:15.957826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:16.026641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ffbd84a77645be87dd5f8fa0086116'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.038019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ffbd84a77645be87dd5f8fa0086116'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.039795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ffbd84a77645be87dd5f8fa0086116'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.046375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:16.056953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ffbd84a77645be87dd5f8fa0086116'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.059209 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ffbd84a77645be87dd5f8fa0086116'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.087410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1194/2385] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:54:15 2022] DELETE /identity/v3/users/26ffbd84a77645be87dd5f8fa0086116 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:16.093660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa332814-b381-4fc0-b00d-2ff20b3e95f8 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:16.093948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa332814-b381-4fc0-b00d-2ff20b3e95f8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:16.094392 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa332814-b381-4fc0-b00d-2ff20b3e95f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:16.117771 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-fa332814-b381-4fc0-b00d-2ff20b3e95f8 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:16.119055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-fa332814-b381-4fc0-b00d-2ff20b3e95f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:16.192375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1192/2386] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:54:16.203348 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:16.204777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:16.269523 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153a62-86c3-47ff-b358-a27e55e9cea4 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:16.274152 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75f747-8ede-479a-bf13-9c08a3e460da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:16.274772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75f747-8ede-479a-bf13-9c08a3e460da service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:16.275045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75f747-8ede-479a-bf13-9c08a3e460da service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:16.275345 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75f747-8ede-479a-bf13-9c08a3e460da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:16.291721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75f747-8ede-479a-bf13-9c08a3e460da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:16.293211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75f747-8ede-479a-bf13-9c08a3e460da service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:16.307847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1195/2387] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:16.651663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa332814-b381-4fc0-b00d-2ff20b3e95f8 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:16.653124 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-fa332814-b381-4fc0-b00d-2ff20b3e95f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:16.667340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa332814-b381-4fc0-b00d-2ff20b3e95f8 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:16.672517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:16.673347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:16.673818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:16.674268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] PATH_INFO: `/v3/projects/1f256e4acd3f4ff9b831685d88a2a179` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:16.675646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f256e4acd3f4ff9b831685d88a2a179)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:16.678134 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:16.784447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f256e4acd3f4ff9b831685d88a2a179', 'request_id': 'req-04ad9d57-bae0-4d56-ab19-cad18f3f311b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.797060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f256e4acd3f4ff9b831685d88a2a179', 'request_id': 'req-04ad9d57-bae0-4d56-ab19-cad18f3f311b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.806696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f256e4acd3f4ff9b831685d88a2a179', 'request_id': 'req-04ad9d57-bae0-4d56-ab19-cad18f3f311b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.821936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] The token cache is being invalidate because project 1f256e4acd3f4ff9b831685d88a2a179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:54:16.822193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.823210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.826716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1193/2388] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:54:16 2022] DELETE /identity/v3/projects/1f256e4acd3f4ff9b831685d88a2a179 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:16.830606 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75f747-8ede-479a-bf13-9c08a3e460da service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:16.841640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75f747-8ede-479a-bf13-9c08a3e460da service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:16.903295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a297dae-c6c8-4155-953f-dc7a6b633bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:16.903924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a297dae-c6c8-4155-953f-dc7a6b633bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:16.904201 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a297dae-c6c8-4155-953f-dc7a6b633bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:16.904473 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a297dae-c6c8-4155-953f-dc7a6b633bd8 admin admin] PATH_INFO: `/v3/users/9340f4cc8bbf49d7b240d26ef8f37dca` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:16.905592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a297dae-c6c8-4155-953f-dc7a6b633bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9340f4cc8bbf49d7b240d26ef8f37dca)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:16.907549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a297dae-c6c8-4155-953f-dc7a6b633bd8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:16.955649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1a297dae-c6c8-4155-953f-dc7a6b633bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9340f4cc8bbf49d7b240d26ef8f37dca'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.964347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1a297dae-c6c8-4155-953f-dc7a6b633bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9340f4cc8bbf49d7b240d26ef8f37dca'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.964977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1a297dae-c6c8-4155-953f-dc7a6b633bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9340f4cc8bbf49d7b240d26ef8f37dca'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.978213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1a297dae-c6c8-4155-953f-dc7a6b633bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9340f4cc8bbf49d7b240d26ef8f37dca'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.978913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1a297dae-c6c8-4155-953f-dc7a6b633bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9340f4cc8bbf49d7b240d26ef8f37dca'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:16.988238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:16.997678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1196/2389] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:54:16 2022] DELETE /identity/v3/users/9340f4cc8bbf49d7b240d26ef8f37dca => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:17.411352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:17.413437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:17.437938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad9d57-bae0-4d56-ab19-cad18f3f311b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:17.476462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:17.515675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441b8d8-cffd-4c49-9369-5188ccc77885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:17.516649 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441b8d8-cffd-4c49-9369-5188ccc77885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:17.516931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441b8d8-cffd-4c49-9369-5188ccc77885 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:17.517233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441b8d8-cffd-4c49-9369-5188ccc77885 admin admin] PATH_INFO: `/v3/projects/31e2015c6d3e4ae8ae44c87c5a984eaa` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:17.519086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441b8d8-cffd-4c49-9369-5188ccc77885 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e2015c6d3e4ae8ae44c87c5a984eaa)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:17.521088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441b8d8-cffd-4c49-9369-5188ccc77885 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:17.577242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7441b8d8-cffd-4c49-9369-5188ccc77885 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e2015c6d3e4ae8ae44c87c5a984eaa', 'request_id': 'req-7441b8d8-cffd-4c49-9369-5188ccc77885'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:17.588451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7441b8d8-cffd-4c49-9369-5188ccc77885 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e2015c6d3e4ae8ae44c87c5a984eaa', 'request_id': 'req-7441b8d8-cffd-4c49-9369-5188ccc77885'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:17.603419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7441b8d8-cffd-4c49-9369-5188ccc77885 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e2015c6d3e4ae8ae44c87c5a984eaa', 'request_id': 'req-7441b8d8-cffd-4c49-9369-5188ccc77885'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:17.624188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7441b8d8-cffd-4c49-9369-5188ccc77885 admin admin] The token cache is being invalidate because project 31e2015c6d3e4ae8ae44c87c5a984eaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:54:17.624711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7441b8d8-cffd-4c49-9369-5188ccc77885 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:17.625791 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7441b8d8-cffd-4c49-9369-5188ccc77885 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:17.628482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1194/2390] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:54:17 2022] DELETE /identity/v3/projects/31e2015c6d3e4ae8ae44c87c5a984eaa => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:17.634547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb0266-d7ba-4d22-83f1-80b795580340 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:17.635027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb0266-d7ba-4d22-83f1-80b795580340 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:17.635221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb0266-d7ba-4d22-83f1-80b795580340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:17.656615 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-51fb0266-d7ba-4d22-83f1-80b795580340 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:17.657830 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-51fb0266-d7ba-4d22-83f1-80b795580340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:17.723408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1197/2391] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:17.731750 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa38559-5e01-4f0b-a55e-96b00a7faaf1 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:17.731946 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa38559-5e01-4f0b-a55e-96b00a7faaf1 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:17.732917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa38559-5e01-4f0b-a55e-96b00a7faaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:17.753755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-daa38559-5e01-4f0b-a55e-96b00a7faaf1 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:17.755438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-daa38559-5e01-4f0b-a55e-96b00a7faaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:17.765591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1195/2392] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:17.769681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb0266-d7ba-4d22-83f1-80b795580340 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:17.780930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb0266-d7ba-4d22-83f1-80b795580340 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:17.785587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ef0e8-1a4e-49ce-a3d3-2e48bb88a1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:17.786184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ef0e8-1a4e-49ce-a3d3-2e48bb88a1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:17.786455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ef0e8-1a4e-49ce-a3d3-2e48bb88a1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:17.786720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ef0e8-1a4e-49ce-a3d3-2e48bb88a1f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:17.787392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ef0e8-1a4e-49ce-a3d3-2e48bb88a1f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:54:17.787687 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ef0e8-1a4e-49ce-a3d3-2e48bb88a1f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:17.790066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ef0e8-1a4e-49ce-a3d3-2e48bb88a1f4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:17.805978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1198/2393] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:54:17.808870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa38559-5e01-4f0b-a55e-96b00a7faaf1 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:17.821880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa38559-5e01-4f0b-a55e-96b00a7faaf1 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:17.828767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac1ba9-c8eb-48ac-b117-843d3912a5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:17.829486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac1ba9-c8eb-48ac-b117-843d3912a5ec admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:17.829840 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac1ba9-c8eb-48ac-b117-843d3912a5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:17.830185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac1ba9-c8eb-48ac-b117-843d3912a5ec admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:17.830917 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac1ba9-c8eb-48ac-b117-843d3912a5ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:17.833160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac1ba9-c8eb-48ac-b117-843d3912a5ec admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:17.873846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1196/2394] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:54:17 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:54:17.878179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ef0e8-1a4e-49ce-a3d3-2e48bb88a1f4 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:17.890953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ef0e8-1a4e-49ce-a3d3-2e48bb88a1f4 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:17.962037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c885b84-294a-49c1-aa81-488db0a93530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:17.962970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c885b84-294a-49c1-aa81-488db0a93530 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:17.963431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c885b84-294a-49c1-aa81-488db0a93530 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:17.963855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c885b84-294a-49c1-aa81-488db0a93530 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:17.964676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c885b84-294a-49c1-aa81-488db0a93530 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:17.966631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c885b84-294a-49c1-aa81-488db0a93530 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:17.993200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1199/2395] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:54:17 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:54:17.997433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac1ba9-c8eb-48ac-b117-843d3912a5ec admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:18.008973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac1ba9-c8eb-48ac-b117-843d3912a5ec admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:18.020663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b1283-4b89-44a2-9b91-ad7abd0f5c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:18.021354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b1283-4b89-44a2-9b91-ad7abd0f5c92 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:18.021698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b1283-4b89-44a2-9b91-ad7abd0f5c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:18.022034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b1283-4b89-44a2-9b91-ad7abd0f5c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:18.022802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b1283-4b89-44a2-9b91-ad7abd0f5c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:18.024562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b1283-4b89-44a2-9b91-ad7abd0f5c92 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:18.035345 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1197/2396] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:18.039809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c885b84-294a-49c1-aa81-488db0a93530 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:18.051237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c885b84-294a-49c1-aa81-488db0a93530 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:18.057117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f9dae-b648-445c-93fd-e840215417ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:18.057715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f9dae-b648-445c-93fd-e840215417ff admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:18.057996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f9dae-b648-445c-93fd-e840215417ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:18.058270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f9dae-b648-445c-93fd-e840215417ff admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:18.058918 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f9dae-b648-445c-93fd-e840215417ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:18.060601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f9dae-b648-445c-93fd-e840215417ff admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:18.073433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1200/2397] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:18.080178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b1283-4b89-44a2-9b91-ad7abd0f5c92 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:18.092582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b1283-4b89-44a2-9b91-ad7abd0f5c92 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:18.097560 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f912ded-fd16-497d-a4d8-cf0157d34fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:18.098246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f912ded-fd16-497d-a4d8-cf0157d34fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:18.098529 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f912ded-fd16-497d-a4d8-cf0157d34fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:18.098806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f912ded-fd16-497d-a4d8-cf0157d34fcc admin admin] PATH_INFO: `/v3/projects/91c4faf78df2401c8050cab8069cc714/users/73c6143ea74741ec9fcb3854d5104f05/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:18.118404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f912ded-fd16-497d-a4d8-cf0157d34fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c4faf78df2401c8050cab8069cc714, user_id=73c6143ea74741ec9fcb3854d5104f05, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:18.121252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f912ded-fd16-497d-a4d8-cf0157d34fcc admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:18.137599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1198/2398] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:18 2022] PUT /identity/v3/projects/91c4faf78df2401c8050cab8069cc714/users/73c6143ea74741ec9fcb3854d5104f05/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:54:18.148987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb26b4f-7005-46fe-81fc-441968b94b3a None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:18.149335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb26b4f-7005-46fe-81fc-441968b94b3a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:18.149530 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb26b4f-7005-46fe-81fc-441968b94b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:18.180006 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-9bb26b4f-7005-46fe-81fc-441968b94b3a None None] MFA Rules not processed for user `73c6143ea74741ec9fcb3854d5104f05`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:18.181489 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9bb26b4f-7005-46fe-81fc-441968b94b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:18.240576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:18.253752 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1201/2399] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:54:18.259464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f912ded-fd16-497d-a4d8-cf0157d34fcc admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:18.321217 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:18.327140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f912ded-fd16-497d-a4d8-cf0157d34fcc admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:18.334465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61713dce-fd02-47d6-b544-5ec89565f4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:18.335373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61713dce-fd02-47d6-b544-5ec89565f4fd admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:18.336522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61713dce-fd02-47d6-b544-5ec89565f4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:18.337064 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61713dce-fd02-47d6-b544-5ec89565f4fd admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:18.337899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61713dce-fd02-47d6-b544-5ec89565f4fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:18.339998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61713dce-fd02-47d6-b544-5ec89565f4fd admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:18.384532 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1199/2400] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:54:18 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:54:18.388081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb26b4f-7005-46fe-81fc-441968b94b3a None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:18.405009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb26b4f-7005-46fe-81fc-441968b94b3a None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:18.479877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbd52e-39dd-4d1d-bad0-4dbe31615bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:18.480724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbd52e-39dd-4d1d-bad0-4dbe31615bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:18.481305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbd52e-39dd-4d1d-bad0-4dbe31615bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:18.481755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbd52e-39dd-4d1d-bad0-4dbe31615bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:18.482413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbd52e-39dd-4d1d-bad0-4dbe31615bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:18.484420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbd52e-39dd-4d1d-bad0-4dbe31615bb4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:18.512462 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1202/2401] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:54:18 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:54:18.513981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61713dce-fd02-47d6-b544-5ec89565f4fd admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:18.525488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61713dce-fd02-47d6-b544-5ec89565f4fd admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:18.530735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db39e01-748c-483e-9a10-2bbcdee5e1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:18.531385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db39e01-748c-483e-9a10-2bbcdee5e1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:18.531666 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db39e01-748c-483e-9a10-2bbcdee5e1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:18.531939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db39e01-748c-483e-9a10-2bbcdee5e1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:18.532585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db39e01-748c-483e-9a10-2bbcdee5e1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:18.534181 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db39e01-748c-483e-9a10-2bbcdee5e1f1 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:18.545766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1200/2402] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:18.549862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbd52e-39dd-4d1d-bad0-4dbe31615bb4 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:18.561410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbd52e-39dd-4d1d-bad0-4dbe31615bb4 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:18.573414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd300b-5421-4536-8528-d4a478ca253d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:18.574240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd300b-5421-4536-8528-d4a478ca253d admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:18.574704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd300b-5421-4536-8528-d4a478ca253d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:18.574980 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd300b-5421-4536-8528-d4a478ca253d admin admin] PATH_INFO: `/v3/projects/f7a86640dfb94acb94f332e56c5f4513/users/347bd932e94f4805bedea2349d634066/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:18.591676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd300b-5421-4536-8528-d4a478ca253d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a86640dfb94acb94f332e56c5f4513, user_id=347bd932e94f4805bedea2349d634066, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:18.594349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd300b-5421-4536-8528-d4a478ca253d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:18.610477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1203/2403] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:18 2022] PUT /identity/v3/projects/f7a86640dfb94acb94f332e56c5f4513/users/347bd932e94f4805bedea2349d634066/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:18.614511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db39e01-748c-483e-9a10-2bbcdee5e1f1 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:18.625755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db39e01-748c-483e-9a10-2bbcdee5e1f1 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:18.697605 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e9d38-2fcb-40b8-a13c-a559b731a01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:18.698376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e9d38-2fcb-40b8-a13c-a559b731a01d admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:18.698725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e9d38-2fcb-40b8-a13c-a559b731a01d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:18.699070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e9d38-2fcb-40b8-a13c-a559b731a01d admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:18.699831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e9d38-2fcb-40b8-a13c-a559b731a01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:18.701535 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e9d38-2fcb-40b8-a13c-a559b731a01d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:18.713684 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1201/2404] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:18.717802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd300b-5421-4536-8528-d4a478ca253d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:18.728861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd300b-5421-4536-8528-d4a478ca253d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:18.733828 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2cf5c-dff9-405e-a6d1-483940779f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:18.734544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2cf5c-dff9-405e-a6d1-483940779f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:18.734889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2cf5c-dff9-405e-a6d1-483940779f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:18.735094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2cf5c-dff9-405e-a6d1-483940779f80 admin admin] PATH_INFO: `/v3/projects/f7a86640dfb94acb94f332e56c5f4513/users/347bd932e94f4805bedea2349d634066/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:18.737268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2cf5c-dff9-405e-a6d1-483940779f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a86640dfb94acb94f332e56c5f4513, user_id=347bd932e94f4805bedea2349d634066, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:18.739985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2cf5c-dff9-405e-a6d1-483940779f80 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:18.759275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1204/2405] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:18 2022] PUT /identity/v3/projects/f7a86640dfb94acb94f332e56c5f4513/users/347bd932e94f4805bedea2349d634066/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:54:18.770449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31a358-ad44-41d5-bc64-7cce92e8fe7a None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:18.770449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31a358-ad44-41d5-bc64-7cce92e8fe7a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:18.770449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31a358-ad44-41d5-bc64-7cce92e8fe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:18.810793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-2d31a358-ad44-41d5-bc64-7cce92e8fe7a None None] MFA Rules not processed for user `347bd932e94f4805bedea2349d634066`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:18.812117 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2d31a358-ad44-41d5-bc64-7cce92e8fe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:18.905829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1202/2406] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:18.918101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2cf5c-dff9-405e-a6d1-483940779f80 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:18.929824 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e1e2cf5c-dff9-405e-a6d1-483940779f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:19.006665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2cf5c-dff9-405e-a6d1-483940779f80 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:19.011837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce0840-4d37-4d5c-9fac-3e34db190db8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:19.012544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce0840-4d37-4d5c-9fac-3e34db190db8 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:19.012896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce0840-4d37-4d5c-9fac-3e34db190db8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:19.013165 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce0840-4d37-4d5c-9fac-3e34db190db8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:19.025046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce0840-4d37-4d5c-9fac-3e34db190db8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:19.026569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce0840-4d37-4d5c-9fac-3e34db190db8 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:19.037741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1205/2407] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:19.101729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31a358-ad44-41d5-bc64-7cce92e8fe7a None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:19.103198 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2d31a358-ad44-41d5-bc64-7cce92e8fe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:19.117634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31a358-ad44-41d5-bc64-7cce92e8fe7a None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:19.121630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee32a7-6eef-4369-b028-2d6893687ab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:19.122517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee32a7-6eef-4369-b028-2d6893687ab2 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:19.122794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee32a7-6eef-4369-b028-2d6893687ab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:19.123087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee32a7-6eef-4369-b028-2d6893687ab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:19.182707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:19.197801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee32a7-6eef-4369-b028-2d6893687ab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:19.199335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee32a7-6eef-4369-b028-2d6893687ab2 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:19.211357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1203/2408] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:22.901613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce0840-4d37-4d5c-9fac-3e34db190db8 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:22.964761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:22.978818 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce0840-4d37-4d5c-9fac-3e34db190db8 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:22.983544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:22.984211 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:22.984489 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:22.984762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] PATH_INFO: `/v3/users/73c6143ea74741ec9fcb3854d5104f05` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:22.985937 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c6143ea74741ec9fcb3854d5104f05)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:22.987911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:23.034694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c6143ea74741ec9fcb3854d5104f05'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:23.042636 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c6143ea74741ec9fcb3854d5104f05'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:23.043296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c6143ea74741ec9fcb3854d5104f05'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:23.056155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c6143ea74741ec9fcb3854d5104f05'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:23.056825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c6143ea74741ec9fcb3854d5104f05'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:23.068969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1206/2409] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:54:22 2022] DELETE /identity/v3/users/73c6143ea74741ec9fcb3854d5104f05 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:23.073474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76700f30-691d-4ebd-a608-222a8be341bb None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:23.073962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76700f30-691d-4ebd-a608-222a8be341bb None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:23.074079 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76700f30-691d-4ebd-a608-222a8be341bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:23.094269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-76700f30-691d-4ebd-a608-222a8be341bb None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:23.095748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-76700f30-691d-4ebd-a608-222a8be341bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:23.163355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1204/2410] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:23.177257 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:23.179422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:23.240517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d033a99-0ba6-4afe-b591-d97c3f8f540d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:23.244972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2394543-d1ae-4b37-be15-ab56d4785b49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:23.245588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2394543-d1ae-4b37-be15-ab56d4785b49 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:23.245867 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2394543-d1ae-4b37-be15-ab56d4785b49 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:23.246270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2394543-d1ae-4b37-be15-ab56d4785b49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:23.256438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2394543-d1ae-4b37-be15-ab56d4785b49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:23.258374 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2394543-d1ae-4b37-be15-ab56d4785b49 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:23.268643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1207/2411] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:23.873791 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76700f30-691d-4ebd-a608-222a8be341bb None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:23.875548 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-76700f30-691d-4ebd-a608-222a8be341bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:23.889393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76700f30-691d-4ebd-a608-222a8be341bb None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:23.894974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:23.896484 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:23.896484 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:23.896484 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] PATH_INFO: `/v3/projects/91c4faf78df2401c8050cab8069cc714` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:23.897811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=91c4faf78df2401c8050cab8069cc714)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:23.900303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:23.973133 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91c4faf78df2401c8050cab8069cc714', 'request_id': 'req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:23.983677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91c4faf78df2401c8050cab8069cc714', 'request_id': 'req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:23.992405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91c4faf78df2401c8050cab8069cc714', 'request_id': 'req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.005163 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] The token cache is being invalidate because project 91c4faf78df2401c8050cab8069cc714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:54:24.005508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.007026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.010948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1205/2412] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:54:23 2022] DELETE /identity/v3/projects/91c4faf78df2401c8050cab8069cc714 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:24.015208 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2394543-d1ae-4b37-be15-ab56d4785b49 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:24.026263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2394543-d1ae-4b37-be15-ab56d4785b49 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:24.102512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9207dc-5e4c-4abf-b79b-96ae9a396d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:24.103819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9207dc-5e4c-4abf-b79b-96ae9a396d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:24.103819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9207dc-5e4c-4abf-b79b-96ae9a396d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:24.103819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9207dc-5e4c-4abf-b79b-96ae9a396d47 admin admin] PATH_INFO: `/v3/users/347bd932e94f4805bedea2349d634066` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:24.105531 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9207dc-5e4c-4abf-b79b-96ae9a396d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=347bd932e94f4805bedea2349d634066)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:24.107611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9207dc-5e4c-4abf-b79b-96ae9a396d47 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:24.123836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:24.181028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6f9207dc-5e4c-4abf-b79b-96ae9a396d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '347bd932e94f4805bedea2349d634066'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.190815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6f9207dc-5e4c-4abf-b79b-96ae9a396d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '347bd932e94f4805bedea2349d634066'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.193231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6f9207dc-5e4c-4abf-b79b-96ae9a396d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '347bd932e94f4805bedea2349d634066'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.210046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6f9207dc-5e4c-4abf-b79b-96ae9a396d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '347bd932e94f4805bedea2349d634066'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.210900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6f9207dc-5e4c-4abf-b79b-96ae9a396d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '347bd932e94f4805bedea2349d634066'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.225019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1208/2413] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:54:24 2022] DELETE /identity/v3/users/347bd932e94f4805bedea2349d634066 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:54:24.732731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:24.734347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:24.750252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843917ff-f7d2-40ec-be0b-4fce84a0a6ba admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:24.817731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182781b-3a6e-4750-bc5c-00409e294a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:24.818370 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182781b-3a6e-4750-bc5c-00409e294a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:24.818644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182781b-3a6e-4750-bc5c-00409e294a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:24.819511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182781b-3a6e-4750-bc5c-00409e294a5a admin admin] PATH_INFO: `/v3/projects/f7a86640dfb94acb94f332e56c5f4513` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:24.820698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182781b-3a6e-4750-bc5c-00409e294a5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a86640dfb94acb94f332e56c5f4513)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:24.822619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182781b-3a6e-4750-bc5c-00409e294a5a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:24.838709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:24.876676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3182781b-3a6e-4750-bc5c-00409e294a5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a86640dfb94acb94f332e56c5f4513', 'request_id': 'req-3182781b-3a6e-4750-bc5c-00409e294a5a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.884989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3182781b-3a6e-4750-bc5c-00409e294a5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a86640dfb94acb94f332e56c5f4513', 'request_id': 'req-3182781b-3a6e-4750-bc5c-00409e294a5a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.892183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3182781b-3a6e-4750-bc5c-00409e294a5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a86640dfb94acb94f332e56c5f4513', 'request_id': 'req-3182781b-3a6e-4750-bc5c-00409e294a5a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.905729 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3182781b-3a6e-4750-bc5c-00409e294a5a admin admin] The token cache is being invalidate because project f7a86640dfb94acb94f332e56c5f4513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:54:24.905929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3182781b-3a6e-4750-bc5c-00409e294a5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.906544 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-3182781b-3a6e-4750-bc5c-00409e294a5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:24.908287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1206/2414] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:54:24 2022] DELETE /identity/v3/projects/f7a86640dfb94acb94f332e56c5f4513 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:24.913129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb724dd-28ab-461d-b276-eb86db4cc71f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:24.913407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb724dd-28ab-461d-b276-eb86db4cc71f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:24.913681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb724dd-28ab-461d-b276-eb86db4cc71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:24.934168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ddb724dd-28ab-461d-b276-eb86db4cc71f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:24.935391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ddb724dd-28ab-461d-b276-eb86db4cc71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:24.997656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1209/2415] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:54:25.005512 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef3ca3-3254-4ec7-9b6e-c503c4d4a071 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.005799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef3ca3-3254-4ec7-9b6e-c503c4d4a071 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.006075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef3ca3-3254-4ec7-9b6e-c503c4d4a071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.030346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-33ef3ca3-3254-4ec7-9b6e-c503c4d4a071 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:25.031708 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-33ef3ca3-3254-4ec7-9b6e-c503c4d4a071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:25.039748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1207/2416] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:25.044140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb724dd-28ab-461d-b276-eb86db4cc71f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:25.054158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb724dd-28ab-461d-b276-eb86db4cc71f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:25.058880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da18bbe-574f-476b-ab46-9be13ed3c18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:25.059632 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da18bbe-574f-476b-ab46-9be13ed3c18d admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.059789 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da18bbe-574f-476b-ab46-9be13ed3c18d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.060055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da18bbe-574f-476b-ab46-9be13ed3c18d admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.060706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da18bbe-574f-476b-ab46-9be13ed3c18d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:54:25.060991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da18bbe-574f-476b-ab46-9be13ed3c18d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:25.062591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da18bbe-574f-476b-ab46-9be13ed3c18d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:25.075614 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1210/2417] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:54:25.079804 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef3ca3-3254-4ec7-9b6e-c503c4d4a071 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:25.090797 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef3ca3-3254-4ec7-9b6e-c503c4d4a071 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:25.096306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa6942-196c-4274-a946-85f6d91322a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:25.097052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa6942-196c-4274-a946-85f6d91322a2 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.097341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa6942-196c-4274-a946-85f6d91322a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.097612 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa6942-196c-4274-a946-85f6d91322a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.098264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa6942-196c-4274-a946-85f6d91322a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:25.100849 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa6942-196c-4274-a946-85f6d91322a2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:25.130554 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1208/2418] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:54:25 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:54:25.135441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da18bbe-574f-476b-ab46-9be13ed3c18d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:25.145546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da18bbe-574f-476b-ab46-9be13ed3c18d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:25.202102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f925125-6d62-4c6e-a624-c972ffaa6ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:25.202701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f925125-6d62-4c6e-a624-c972ffaa6ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.202974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f925125-6d62-4c6e-a624-c972ffaa6ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.203273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f925125-6d62-4c6e-a624-c972ffaa6ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.203898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f925125-6d62-4c6e-a624-c972ffaa6ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:25.205813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f925125-6d62-4c6e-a624-c972ffaa6ca6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:25.227262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1211/2419] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:54:25 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:54:25.231655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa6942-196c-4274-a946-85f6d91322a2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:25.243295 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa6942-196c-4274-a946-85f6d91322a2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:25.248673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec0a46-57a8-4b9a-8caf-c03ee9d5f411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:25.249482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec0a46-57a8-4b9a-8caf-c03ee9d5f411 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.249877 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec0a46-57a8-4b9a-8caf-c03ee9d5f411 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.250160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec0a46-57a8-4b9a-8caf-c03ee9d5f411 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.250846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec0a46-57a8-4b9a-8caf-c03ee9d5f411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:25.252510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec0a46-57a8-4b9a-8caf-c03ee9d5f411 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:25.263592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1209/2420] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:25.268141 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f925125-6d62-4c6e-a624-c972ffaa6ca6 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:25.278431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f925125-6d62-4c6e-a624-c972ffaa6ca6 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:25.290780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf4118-c012-4ad2-9646-22b8f52e1089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:25.291420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf4118-c012-4ad2-9646-22b8f52e1089 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.291695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf4118-c012-4ad2-9646-22b8f52e1089 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.291961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf4118-c012-4ad2-9646-22b8f52e1089 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.292599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf4118-c012-4ad2-9646-22b8f52e1089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:25.294228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf4118-c012-4ad2-9646-22b8f52e1089 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:25.296745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:25.308226 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1212/2421] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:25.312595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec0a46-57a8-4b9a-8caf-c03ee9d5f411 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:25.323618 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec0a46-57a8-4b9a-8caf-c03ee9d5f411 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:25.329217 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8af52e-f280-4929-b0d3-6f29d5f4f70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:25.329890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8af52e-f280-4929-b0d3-6f29d5f4f70c admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.330172 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8af52e-f280-4929-b0d3-6f29d5f4f70c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.330446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8af52e-f280-4929-b0d3-6f29d5f4f70c admin admin] PATH_INFO: `/v3/projects/51e2d7943d0045c08c9da4b50bc248d0/users/1c768a5a7aee43658a64fa1af2a809c6/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.347520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8af52e-f280-4929-b0d3-6f29d5f4f70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e2d7943d0045c08c9da4b50bc248d0, user_id=1c768a5a7aee43658a64fa1af2a809c6, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:25.350858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8af52e-f280-4929-b0d3-6f29d5f4f70c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:25.365468 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1210/2422] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:25 2022] PUT /identity/v3/projects/51e2d7943d0045c08c9da4b50bc248d0/users/1c768a5a7aee43658a64fa1af2a809c6/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:25.377910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdf4fc-39e2-4385-b6ec-8ea51b15a962 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.378191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdf4fc-39e2-4385-b6ec-8ea51b15a962 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.378461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdf4fc-39e2-4385-b6ec-8ea51b15a962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.412021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-7dbdf4fc-39e2-4385-b6ec-8ea51b15a962 None None] MFA Rules not processed for user `1c768a5a7aee43658a64fa1af2a809c6`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:25.414085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7dbdf4fc-39e2-4385-b6ec-8ea51b15a962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:25.482745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1213/2423] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:54:25.487350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8af52e-f280-4929-b0d3-6f29d5f4f70c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:25.553703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8af52e-f280-4929-b0d3-6f29d5f4f70c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:25.558830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e1b41-00b0-4dbf-a603-461bd7f9b6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:25.559473 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e1b41-00b0-4dbf-a603-461bd7f9b6ec admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.559748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e1b41-00b0-4dbf-a603-461bd7f9b6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.560019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e1b41-00b0-4dbf-a603-461bd7f9b6ec admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.560666 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e1b41-00b0-4dbf-a603-461bd7f9b6ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:25.562569 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e1b41-00b0-4dbf-a603-461bd7f9b6ec admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:25.591438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1211/2424] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:54:25 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:54:25.595379 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdf4fc-39e2-4385-b6ec-8ea51b15a962 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:25.606897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdf4fc-39e2-4385-b6ec-8ea51b15a962 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:25.668276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76795ce9-e201-4e6e-82b6-b75feff428f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:25.668878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76795ce9-e201-4e6e-82b6-b75feff428f1 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.669148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76795ce9-e201-4e6e-82b6-b75feff428f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.669414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76795ce9-e201-4e6e-82b6-b75feff428f1 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.670046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76795ce9-e201-4e6e-82b6-b75feff428f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:25.672067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76795ce9-e201-4e6e-82b6-b75feff428f1 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:25.696554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1214/2425] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:54:25 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:54:25.698504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e1b41-00b0-4dbf-a603-461bd7f9b6ec admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:25.709652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e1b41-00b0-4dbf-a603-461bd7f9b6ec admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:25.714742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6788d-4fa4-40bc-b5ac-dd6f656382e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:25.715381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6788d-4fa4-40bc-b5ac-dd6f656382e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.715663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6788d-4fa4-40bc-b5ac-dd6f656382e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.716199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6788d-4fa4-40bc-b5ac-dd6f656382e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.716850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6788d-4fa4-40bc-b5ac-dd6f656382e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:25.718881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6788d-4fa4-40bc-b5ac-dd6f656382e0 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:25.730599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1212/2426] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:25.735021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76795ce9-e201-4e6e-82b6-b75feff428f1 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:25.745868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76795ce9-e201-4e6e-82b6-b75feff428f1 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:25.750935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ebe0f-9eb8-4fe5-a0f2-a510b0825401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:25.751623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ebe0f-9eb8-4fe5-a0f2-a510b0825401 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.751901 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ebe0f-9eb8-4fe5-a0f2-a510b0825401 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.752169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ebe0f-9eb8-4fe5-a0f2-a510b0825401 admin admin] PATH_INFO: `/v3/projects/f00a5c475bf44c4ebf238af33907d665/users/42a5a79720fa4f4fa4627758bfc20c06/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.771186 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ebe0f-9eb8-4fe5-a0f2-a510b0825401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00a5c475bf44c4ebf238af33907d665, user_id=42a5a79720fa4f4fa4627758bfc20c06, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:25.773919 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ebe0f-9eb8-4fe5-a0f2-a510b0825401 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:25.787655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1215/2427] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:25 2022] PUT /identity/v3/projects/f00a5c475bf44c4ebf238af33907d665/users/42a5a79720fa4f4fa4627758bfc20c06/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:25.791852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6788d-4fa4-40bc-b5ac-dd6f656382e0 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:25.801648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6788d-4fa4-40bc-b5ac-dd6f656382e0 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:25.812466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:25.877264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4c0a0-3e89-4efe-9116-97bf81f4dd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:25.877973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4c0a0-3e89-4efe-9116-97bf81f4dd49 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.878329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4c0a0-3e89-4efe-9116-97bf81f4dd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.878674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4c0a0-3e89-4efe-9116-97bf81f4dd49 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.879543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4c0a0-3e89-4efe-9116-97bf81f4dd49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:25.881311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4c0a0-3e89-4efe-9116-97bf81f4dd49 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:25.895383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1213/2428] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:25.896535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ebe0f-9eb8-4fe5-a0f2-a510b0825401 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:25.907277 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ebe0f-9eb8-4fe5-a0f2-a510b0825401 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:25.912065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad28a7d-190e-4e3d-a735-62d59a90e909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:25.913326 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad28a7d-190e-4e3d-a735-62d59a90e909 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.913618 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad28a7d-190e-4e3d-a735-62d59a90e909 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.913866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad28a7d-190e-4e3d-a735-62d59a90e909 admin admin] PATH_INFO: `/v3/projects/f00a5c475bf44c4ebf238af33907d665/users/42a5a79720fa4f4fa4627758bfc20c06/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.915944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad28a7d-190e-4e3d-a735-62d59a90e909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00a5c475bf44c4ebf238af33907d665, user_id=42a5a79720fa4f4fa4627758bfc20c06, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:25.919442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad28a7d-190e-4e3d-a735-62d59a90e909 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:25.932025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1216/2429] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:25 2022] PUT /identity/v3/projects/f00a5c475bf44c4ebf238af33907d665/users/42a5a79720fa4f4fa4627758bfc20c06/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:25.942641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed64e2-226f-4285-a374-47017272c2f7 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:25.942927 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed64e2-226f-4285-a374-47017272c2f7 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:25.943235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed64e2-226f-4285-a374-47017272c2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:25.974239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-64ed64e2-226f-4285-a374-47017272c2f7 None None] MFA Rules not processed for user `42a5a79720fa4f4fa4627758bfc20c06`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:25.976089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-64ed64e2-226f-4285-a374-47017272c2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:26.062862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1214/2430] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:26.067790 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad28a7d-190e-4e3d-a735-62d59a90e909 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:26.070270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5ad28a7d-190e-4e3d-a735-62d59a90e909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:26.118492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:26.134739 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad28a7d-190e-4e3d-a735-62d59a90e909 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:26.138346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1580207-0f6e-46f2-906e-3a5e2fb08f4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:26.138944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1580207-0f6e-46f2-906e-3a5e2fb08f4c service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:26.139252 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1580207-0f6e-46f2-906e-3a5e2fb08f4c service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:26.139520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1580207-0f6e-46f2-906e-3a5e2fb08f4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:26.204438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1580207-0f6e-46f2-906e-3a5e2fb08f4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:26.205873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1580207-0f6e-46f2-906e-3a5e2fb08f4c service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:26.215834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1217/2431] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:54:28.035028 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed64e2-226f-4285-a374-47017272c2f7 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:28.036892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-64ed64e2-226f-4285-a374-47017272c2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:28.204595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed64e2-226f-4285-a374-47017272c2f7 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:28.211706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099bfa3-5921-4fcc-bbde-4419e131e05b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:28.212474 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099bfa3-5921-4fcc-bbde-4419e131e05b service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:28.213176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099bfa3-5921-4fcc-bbde-4419e131e05b service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:28.213176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099bfa3-5921-4fcc-bbde-4419e131e05b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:28.216866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9099bfa3-5921-4fcc-bbde-4419e131e05b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:28.230745 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099bfa3-5921-4fcc-bbde-4419e131e05b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:28.232733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099bfa3-5921-4fcc-bbde-4419e131e05b service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:28.246085 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1215/2432] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:28.795254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1580207-0f6e-46f2-906e-3a5e2fb08f4c service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:28.808340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1580207-0f6e-46f2-906e-3a5e2fb08f4c service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:28.813994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48597f65-19ca-4cd5-9505-1f3a3bbd3a77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:28.814673 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48597f65-19ca-4cd5-9505-1f3a3bbd3a77 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:28.815010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48597f65-19ca-4cd5-9505-1f3a3bbd3a77 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:28.815519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48597f65-19ca-4cd5-9505-1f3a3bbd3a77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:28.817269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-48597f65-19ca-4cd5-9505-1f3a3bbd3a77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:28.830632 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48597f65-19ca-4cd5-9505-1f3a3bbd3a77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:28.832491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48597f65-19ca-4cd5-9505-1f3a3bbd3a77 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:28.843794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1218/2433] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:30.823371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099bfa3-5921-4fcc-bbde-4419e131e05b service nova] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:30.824894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9099bfa3-5921-4fcc-bbde-4419e131e05b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:30.836936 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099bfa3-5921-4fcc-bbde-4419e131e05b service nova] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:30.840985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0f264-dc9a-403f-88c7-6f59d24957e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:30.841599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0f264-dc9a-403f-88c7-6f59d24957e3 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:30.841881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0f264-dc9a-403f-88c7-6f59d24957e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:30.842159 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0f264-dc9a-403f-88c7-6f59d24957e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:30.864374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0f264-dc9a-403f-88c7-6f59d24957e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:30.867308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0f264-dc9a-403f-88c7-6f59d24957e3 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:30.879841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1216/2434] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:36.027482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48597f65-19ca-4cd5-9505-1f3a3bbd3a77 service nova] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:36.091649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48597f65-19ca-4cd5-9505-1f3a3bbd3a77 service nova] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:36.096137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:36.096761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:36.097035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:36.097296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] PATH_INFO: `/v3/users/1c768a5a7aee43658a64fa1af2a809c6` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:36.098399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c768a5a7aee43658a64fa1af2a809c6)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:36.100387 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:36.147367 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c768a5a7aee43658a64fa1af2a809c6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.155885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c768a5a7aee43658a64fa1af2a809c6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.156492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c768a5a7aee43658a64fa1af2a809c6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.169013 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c768a5a7aee43658a64fa1af2a809c6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.169679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c768a5a7aee43658a64fa1af2a809c6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.178030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:36.185040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1219/2435] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:54:36 2022] DELETE /identity/v3/users/1c768a5a7aee43658a64fa1af2a809c6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:36.189777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097951e-f2b9-4866-930f-ad79cc0b32f5 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:36.190271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097951e-f2b9-4866-930f-ad79cc0b32f5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:36.190402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097951e-f2b9-4866-930f-ad79cc0b32f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:36.208168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:36.213182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-7097951e-f2b9-4866-930f-ad79cc0b32f5 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:36.214429 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7097951e-f2b9-4866-930f-ad79cc0b32f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:36.281145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1217/2436] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:36.292120 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:36.293867 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:36.359210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750392e-688c-49be-ab24-5526efb5d0eb admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:36.363005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e84b77-02a5-4efb-b376-565d4a63b38d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:36.363650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e84b77-02a5-4efb-b376-565d4a63b38d service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:36.363933 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e84b77-02a5-4efb-b376-565d4a63b38d service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:36.364204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e84b77-02a5-4efb-b376-565d4a63b38d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:36.374609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e84b77-02a5-4efb-b376-565d4a63b38d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:36.376212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e84b77-02a5-4efb-b376-565d4a63b38d service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:36.386944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1220/2437] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:36.674088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097951e-f2b9-4866-930f-ad79cc0b32f5 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:36.675632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7097951e-f2b9-4866-930f-ad79cc0b32f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:36.689148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097951e-f2b9-4866-930f-ad79cc0b32f5 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:36.695041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:36.695852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:36.696136 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:36.696412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] PATH_INFO: `/v3/projects/51e2d7943d0045c08c9da4b50bc248d0` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:36.697572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51e2d7943d0045c08c9da4b50bc248d0)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:36.699517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:36.748617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51e2d7943d0045c08c9da4b50bc248d0', 'request_id': 'req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.755891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51e2d7943d0045c08c9da4b50bc248d0', 'request_id': 'req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.762878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51e2d7943d0045c08c9da4b50bc248d0', 'request_id': 'req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.778426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] The token cache is being invalidate because project 51e2d7943d0045c08c9da4b50bc248d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:54:36.778929 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.779944 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.781471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1218/2438] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:54:36 2022] DELETE /identity/v3/projects/51e2d7943d0045c08c9da4b50bc248d0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:36.785427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e84b77-02a5-4efb-b376-565d4a63b38d service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:36.795347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e84b77-02a5-4efb-b376-565d4a63b38d service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:36.852233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f449ad-479f-4a1b-8795-3ee0884821b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:36.852863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f449ad-479f-4a1b-8795-3ee0884821b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:36.853143 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f449ad-479f-4a1b-8795-3ee0884821b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:36.853415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f449ad-479f-4a1b-8795-3ee0884821b0 admin admin] PATH_INFO: `/v3/users/42a5a79720fa4f4fa4627758bfc20c06` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:36.854584 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f449ad-479f-4a1b-8795-3ee0884821b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42a5a79720fa4f4fa4627758bfc20c06)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:36.856542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f449ad-479f-4a1b-8795-3ee0884821b0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:36.903066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e2f449ad-479f-4a1b-8795-3ee0884821b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42a5a79720fa4f4fa4627758bfc20c06'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.910183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e2f449ad-479f-4a1b-8795-3ee0884821b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42a5a79720fa4f4fa4627758bfc20c06'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.910795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e2f449ad-479f-4a1b-8795-3ee0884821b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42a5a79720fa4f4fa4627758bfc20c06'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.922880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e2f449ad-479f-4a1b-8795-3ee0884821b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42a5a79720fa4f4fa4627758bfc20c06'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.923544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e2f449ad-479f-4a1b-8795-3ee0884821b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42a5a79720fa4f4fa4627758bfc20c06'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:36.936062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1221/2439] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:54:36 2022] DELETE /identity/v3/users/42a5a79720fa4f4fa4627758bfc20c06 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:37.216948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:37.218507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:37.232681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f8cb2-fe4c-4757-8cd4-a9eca54598e7 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:37.294335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a88611-bde1-4ab7-b515-6454436af1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:37.294952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a88611-bde1-4ab7-b515-6454436af1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:37.295398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a88611-bde1-4ab7-b515-6454436af1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:37.295679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a88611-bde1-4ab7-b515-6454436af1b4 admin admin] PATH_INFO: `/v3/projects/f00a5c475bf44c4ebf238af33907d665` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:37.296958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a88611-bde1-4ab7-b515-6454436af1b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00a5c475bf44c4ebf238af33907d665)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:37.298885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a88611-bde1-4ab7-b515-6454436af1b4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:37.337775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:37.353855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-70a88611-bde1-4ab7-b515-6454436af1b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00a5c475bf44c4ebf238af33907d665', 'request_id': 'req-70a88611-bde1-4ab7-b515-6454436af1b4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:37.362213 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-70a88611-bde1-4ab7-b515-6454436af1b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00a5c475bf44c4ebf238af33907d665', 'request_id': 'req-70a88611-bde1-4ab7-b515-6454436af1b4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:37.369519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-70a88611-bde1-4ab7-b515-6454436af1b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00a5c475bf44c4ebf238af33907d665', 'request_id': 'req-70a88611-bde1-4ab7-b515-6454436af1b4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:37.380642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-70a88611-bde1-4ab7-b515-6454436af1b4 admin admin] The token cache is being invalidate because project f00a5c475bf44c4ebf238af33907d665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:54:37.381009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-70a88611-bde1-4ab7-b515-6454436af1b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:37.381823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-70a88611-bde1-4ab7-b515-6454436af1b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:37.383464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1219/2440] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:54:37 2022] DELETE /identity/v3/projects/f00a5c475bf44c4ebf238af33907d665 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:37.389349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88487d38-a33f-4292-8be8-0985c819e5fe None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:37.389795 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88487d38-a33f-4292-8be8-0985c819e5fe None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:37.389987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88487d38-a33f-4292-8be8-0985c819e5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:37.415764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-88487d38-a33f-4292-8be8-0985c819e5fe None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:37.416987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-88487d38-a33f-4292-8be8-0985c819e5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:37.435822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:37.483341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1222/2441] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:54:37.492859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec2c4a-ee34-430b-b66f-06f7e3051b86 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:37.493147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec2c4a-ee34-430b-b66f-06f7e3051b86 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:37.493427 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec2c4a-ee34-430b-b66f-06f7e3051b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:37.514806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-afec2c4a-ee34-430b-b66f-06f7e3051b86 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:37.516047 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-afec2c4a-ee34-430b-b66f-06f7e3051b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:37.525109 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1220/2442] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:37.529458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88487d38-a33f-4292-8be8-0985c819e5fe None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:37.539938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88487d38-a33f-4292-8be8-0985c819e5fe None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:37.544732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de69d70-98c8-4dfa-b94b-0c58b741f02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:37.545331 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de69d70-98c8-4dfa-b94b-0c58b741f02c admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:37.545608 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de69d70-98c8-4dfa-b94b-0c58b741f02c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:37.545873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de69d70-98c8-4dfa-b94b-0c58b741f02c admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:37.546541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de69d70-98c8-4dfa-b94b-0c58b741f02c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:54:37.546838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de69d70-98c8-4dfa-b94b-0c58b741f02c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:37.548467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de69d70-98c8-4dfa-b94b-0c58b741f02c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:37.560394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1223/2443] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:54:37.564879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec2c4a-ee34-430b-b66f-06f7e3051b86 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:37.574847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec2c4a-ee34-430b-b66f-06f7e3051b86 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:37.580325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52d70-68ba-4350-9587-2554e650ddab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:37.580925 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52d70-68ba-4350-9587-2554e650ddab admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:37.581303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52d70-68ba-4350-9587-2554e650ddab admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:37.581477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52d70-68ba-4350-9587-2554e650ddab admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:37.582981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c52d70-68ba-4350-9587-2554e650ddab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:37.584185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c52d70-68ba-4350-9587-2554e650ddab admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:37.614866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1221/2444] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:54:37 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:54:37.619413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de69d70-98c8-4dfa-b94b-0c58b741f02c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:37.631241 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de69d70-98c8-4dfa-b94b-0c58b741f02c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:37.692892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a53726-a3fa-4ba0-b37a-a2686dad171f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:37.693501 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a53726-a3fa-4ba0-b37a-a2686dad171f admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:37.693780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a53726-a3fa-4ba0-b37a-a2686dad171f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:37.694123 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a53726-a3fa-4ba0-b37a-a2686dad171f admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:37.694755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a53726-a3fa-4ba0-b37a-a2686dad171f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:37.696727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a53726-a3fa-4ba0-b37a-a2686dad171f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:37.727111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1224/2445] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:54:37 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:54:37.731545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52d70-68ba-4350-9587-2554e650ddab admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:37.743006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52d70-68ba-4350-9587-2554e650ddab admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:37.748168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9eb91-56e4-456f-a4eb-9a1843a6d106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:37.749152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9eb91-56e4-456f-a4eb-9a1843a6d106 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:37.749644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9eb91-56e4-456f-a4eb-9a1843a6d106 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:37.749789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9eb91-56e4-456f-a4eb-9a1843a6d106 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:37.750575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9eb91-56e4-456f-a4eb-9a1843a6d106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:37.752277 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9eb91-56e4-456f-a4eb-9a1843a6d106 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:37.767426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1222/2446] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:37.768968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a53726-a3fa-4ba0-b37a-a2686dad171f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:37.780423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a53726-a3fa-4ba0-b37a-a2686dad171f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:37.785543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26b3b4-d484-49d9-83b6-d3a73e33f8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:37.786139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26b3b4-d484-49d9-83b6-d3a73e33f8ab admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:37.786545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26b3b4-d484-49d9-83b6-d3a73e33f8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:37.786712 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26b3b4-d484-49d9-83b6-d3a73e33f8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:37.787327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26b3b4-d484-49d9-83b6-d3a73e33f8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:37.789843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26b3b4-d484-49d9-83b6-d3a73e33f8ab admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:37.801115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1225/2447] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:37.806034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9eb91-56e4-456f-a4eb-9a1843a6d106 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:37.816680 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9eb91-56e4-456f-a4eb-9a1843a6d106 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:37.823833 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e721192-65fe-4325-a96f-6d0b23941701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:37.824507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e721192-65fe-4325-a96f-6d0b23941701 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:37.824807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e721192-65fe-4325-a96f-6d0b23941701 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:37.825084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e721192-65fe-4325-a96f-6d0b23941701 admin admin] PATH_INFO: `/v3/projects/7713ff3ccc8546b68a89f2e2087d5abf/users/1530c2946ff3486daf7c8b2b70c1a3b6/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:37.840938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e721192-65fe-4325-a96f-6d0b23941701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7713ff3ccc8546b68a89f2e2087d5abf, user_id=1530c2946ff3486daf7c8b2b70c1a3b6, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:37.843666 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e721192-65fe-4325-a96f-6d0b23941701 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:37.857691 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1223/2448] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:37 2022] PUT /identity/v3/projects/7713ff3ccc8546b68a89f2e2087d5abf/users/1530c2946ff3486daf7c8b2b70c1a3b6/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:37.862813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6f3ec-8a1e-4baa-b0f9-2fb22fa43cf4 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:37.863202 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6f3ec-8a1e-4baa-b0f9-2fb22fa43cf4 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:37.863435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6f3ec-8a1e-4baa-b0f9-2fb22fa43cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:37.883403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ffd6f3ec-8a1e-4baa-b0f9-2fb22fa43cf4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:37.884633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ffd6f3ec-8a1e-4baa-b0f9-2fb22fa43cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:37.943458 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1226/2449] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:37.953981 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e721192-65fe-4325-a96f-6d0b23941701 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:37.955723 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7e721192-65fe-4325-a96f-6d0b23941701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:38.016044 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e721192-65fe-4325-a96f-6d0b23941701 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:38.019797 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c71eaa-8e1e-4bdb-b4c2-09c1fc534a00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:38.020400 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c71eaa-8e1e-4bdb-b4c2-09c1fc534a00 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:38.020680 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c71eaa-8e1e-4bdb-b4c2-09c1fc534a00 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:38.020953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c71eaa-8e1e-4bdb-b4c2-09c1fc534a00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:38.030926 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c71eaa-8e1e-4bdb-b4c2-09c1fc534a00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:38.032390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c71eaa-8e1e-4bdb-b4c2-09c1fc534a00 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:38.042082 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1224/2450] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:38.942562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccef0fe-045e-4703-a648-fcf3f3b2c93c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:38.943061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccef0fe-045e-4703-a648-fcf3f3b2c93c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:38.943206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccef0fe-045e-4703-a648-fcf3f3b2c93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:38.975810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-eccef0fe-045e-4703-a648-fcf3f3b2c93c None None] MFA Rules not processed for user `1530c2946ff3486daf7c8b2b70c1a3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:38.977512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-eccef0fe-045e-4703-a648-fcf3f3b2c93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:38.998785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:39.082543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1227/2451] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:54:39.095652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c71eaa-8e1e-4bdb-b4c2-09c1fc534a00 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:39.097196 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-16c71eaa-8e1e-4bdb-b4c2-09c1fc534a00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:39.170982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c71eaa-8e1e-4bdb-b4c2-09c1fc534a00 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:39.179177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b771a-e5ba-4143-9aa9-15875124b9bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:39.179177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b771a-e5ba-4143-9aa9-15875124b9bb service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:39.179177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b771a-e5ba-4143-9aa9-15875124b9bb service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:39.179177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b771a-e5ba-4143-9aa9-15875124b9bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:39.204657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b771a-e5ba-4143-9aa9-15875124b9bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:39.206561 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b771a-e5ba-4143-9aa9-15875124b9bb service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:39.219186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1225/2452] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:40.610586 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096e176-9c09-4f3b-a9b8-a54dcb7a4b77 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:40.611257 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096e176-9c09-4f3b-a9b8-a54dcb7a4b77 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:40.611257 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096e176-9c09-4f3b-a9b8-a54dcb7a4b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:40.633026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a096e176-9c09-4f3b-a9b8-a54dcb7a4b77 None None] MFA Rules not processed for user `1530c2946ff3486daf7c8b2b70c1a3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:40.634248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a096e176-9c09-4f3b-a9b8-a54dcb7a4b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:40.641208 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1228/2453] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:40.882627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b771a-e5ba-4143-9aa9-15875124b9bb service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:40.886883 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:40.893570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b771a-e5ba-4143-9aa9-15875124b9bb service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:40.898269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3737216-a4df-4c0f-90af-f1b2620c8697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:40.898866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3737216-a4df-4c0f-90af-f1b2620c8697 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:40.899171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3737216-a4df-4c0f-90af-f1b2620c8697 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:40.899448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3737216-a4df-4c0f-90af-f1b2620c8697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:40.901202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b3737216-a4df-4c0f-90af-f1b2620c8697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:40.911387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3737216-a4df-4c0f-90af-f1b2620c8697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:40.913543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3737216-a4df-4c0f-90af-f1b2620c8697 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:40.921819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1226/2454] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:50.626241 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=22 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:50.918023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=5 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:57.339195 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62e8ab-1e3c-46ba-9408-f8a27ad839ab None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:57.339748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62e8ab-1e3c-46ba-9408-f8a27ad839ab None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:57.339748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62e8ab-1e3c-46ba-9408-f8a27ad839ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:57.363278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-3b62e8ab-1e3c-46ba-9408-f8a27ad839ab None None] MFA Rules not processed for user `1530c2946ff3486daf7c8b2b70c1a3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:57.363278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3b62e8ab-1e3c-46ba-9408-f8a27ad839ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:57.370088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1229/2455] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:54:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:03.632523 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3737216-a4df-4c0f-90af-f1b2620c8697 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:03.634096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b3737216-a4df-4c0f-90af-f1b2620c8697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:03.644428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3737216-a4df-4c0f-90af-f1b2620c8697 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:03.648038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9729a24-13ff-4088-bb0a-91b33c114a90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:03.649129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9729a24-13ff-4088-bb0a-91b33c114a90 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:03.649401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9729a24-13ff-4088-bb0a-91b33c114a90 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:03.649664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9729a24-13ff-4088-bb0a-91b33c114a90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:03.651326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d9729a24-13ff-4088-bb0a-91b33c114a90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:03.663404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9729a24-13ff-4088-bb0a-91b33c114a90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:03.665489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9729a24-13ff-4088-bb0a-91b33c114a90 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:03.675183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1227/2456] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:05.956939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8ff7a-ef85-4240-8dfa-732d6bc84061 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:05.957385 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8ff7a-ef85-4240-8dfa-732d6bc84061 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:05.957484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8ff7a-ef85-4240-8dfa-732d6bc84061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:05.978506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-64d8ff7a-ef85-4240-8dfa-732d6bc84061 None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:05.979722 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-64d8ff7a-ef85-4240-8dfa-732d6bc84061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:06.043093 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1230/2457] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:55:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:06.111946 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9729a24-13ff-4088-bb0a-91b33c114a90 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:06.113458 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d9729a24-13ff-4088-bb0a-91b33c114a90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:06.201626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9729a24-13ff-4088-bb0a-91b33c114a90 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:06.207391 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98653c-b4b4-44af-8502-10926361f07e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:06.208028 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98653c-b4b4-44af-8502-10926361f07e service swift] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.208331 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98653c-b4b4-44af-8502-10926361f07e service swift] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.208608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98653c-b4b4-44af-8502-10926361f07e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.219950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98653c-b4b4-44af-8502-10926361f07e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:06.222987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98653c-b4b4-44af-8502-10926361f07e service swift] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:06.236493 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1228/2458] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:55:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:55:08.089641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8ff7a-ef85-4240-8dfa-732d6bc84061 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:08.091767 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-64d8ff7a-ef85-4240-8dfa-732d6bc84061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:08.119238 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8ff7a-ef85-4240-8dfa-732d6bc84061 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:08.125263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:08.125977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:08.126349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:08.126742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] PATH_INFO: `/v3/users/b6ca4276e1554d4184451313b2e09a88` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:08.129111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ca4276e1554d4184451313b2e09a88)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:08.131668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:08.199016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ca4276e1554d4184451313b2e09a88'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:08.208609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ca4276e1554d4184451313b2e09a88'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:08.209479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ca4276e1554d4184451313b2e09a88'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:08.224324 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ca4276e1554d4184451313b2e09a88'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:08.225527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ca4276e1554d4184451313b2e09a88'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:08.243727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1231/2459] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:08 2022] DELETE /identity/v3/users/b6ca4276e1554d4184451313b2e09a88 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:08.486632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98653c-b4b4-44af-8502-10926361f07e service swift] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:08.488291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1b98653c-b4b4-44af-8502-10926361f07e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:08.591280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:08.593562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:08.620461 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98653c-b4b4-44af-8502-10926361f07e service swift] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:08.719365 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c076f02-2c78-430e-b71a-4bd3cf8b7f18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:08.719986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c076f02-2c78-430e-b71a-4bd3cf8b7f18 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:08.720267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c076f02-2c78-430e-b71a-4bd3cf8b7f18 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:08.720540 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c076f02-2c78-430e-b71a-4bd3cf8b7f18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:08.722479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3c076f02-2c78-430e-b71a-4bd3cf8b7f18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:08.732297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3afb6c-9ee2-4536-9781-fb20dd369754 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:08.737637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c721c-a969-423f-b519-1fe7ec09c081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:08.739236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c721c-a969-423f-b519-1fe7ec09c081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:08.739709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c721c-a969-423f-b519-1fe7ec09c081 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:08.740258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c721c-a969-423f-b519-1fe7ec09c081 admin admin] PATH_INFO: `/v3/projects/568beb0822e640c1bacd98a2aec950ff` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:08.741416 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c721c-a969-423f-b519-1fe7ec09c081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=568beb0822e640c1bacd98a2aec950ff)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:08.743695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:08.748262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c721c-a969-423f-b519-1fe7ec09c081 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:08.768136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:08.821953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c076f02-2c78-430e-b71a-4bd3cf8b7f18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:08.824838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c076f02-2c78-430e-b71a-4bd3cf8b7f18 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:08.832189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-119c721c-a969-423f-b519-1fe7ec09c081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568beb0822e640c1bacd98a2aec950ff', 'request_id': 'req-119c721c-a969-423f-b519-1fe7ec09c081'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:08.848658 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-119c721c-a969-423f-b519-1fe7ec09c081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568beb0822e640c1bacd98a2aec950ff', 'request_id': 'req-119c721c-a969-423f-b519-1fe7ec09c081'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:08.855412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1229/2460] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:08.862428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-119c721c-a969-423f-b519-1fe7ec09c081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568beb0822e640c1bacd98a2aec950ff', 'request_id': 'req-119c721c-a969-423f-b519-1fe7ec09c081'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:08.888088 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-119c721c-a969-423f-b519-1fe7ec09c081 admin admin] The token cache is being invalidate because project 568beb0822e640c1bacd98a2aec950ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:08.888465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-119c721c-a969-423f-b519-1fe7ec09c081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:08.889170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-119c721c-a969-423f-b519-1fe7ec09c081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:08.898504 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1232/2461] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:08 2022] DELETE /identity/v3/projects/568beb0822e640c1bacd98a2aec950ff => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:55:08.900326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf516f0-6ba2-4f6a-b2ff-84b886b53ead None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:08.900645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf516f0-6ba2-4f6a-b2ff-84b886b53ead None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:08.901017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf516f0-6ba2-4f6a-b2ff-84b886b53ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:08.944668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0cf516f0-6ba2-4f6a-b2ff-84b886b53ead None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:08.946734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0cf516f0-6ba2-4f6a-b2ff-84b886b53ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:09.059904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1230/2462] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:09.064264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fb12c-5fb9-4f04-a432-edb48f1ce52f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:09.064637 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fb12c-5fb9-4f04-a432-edb48f1ce52f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:09.064982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fb12c-5fb9-4f04-a432-edb48f1ce52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:09.105871 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-150fb12c-5fb9-4f04-a432-edb48f1ce52f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:09.106444 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-150fb12c-5fb9-4f04-a432-edb48f1ce52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:09.122580 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1233/2463] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:09.126602 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf516f0-6ba2-4f6a-b2ff-84b886b53ead None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:09.137294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf516f0-6ba2-4f6a-b2ff-84b886b53ead None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:09.142187 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce220b-a37a-4ebf-8bd6-72426127d0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:09.142794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce220b-a37a-4ebf-8bd6-72426127d0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:09.143075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce220b-a37a-4ebf-8bd6-72426127d0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:09.143388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce220b-a37a-4ebf-8bd6-72426127d0a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:09.144177 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce220b-a37a-4ebf-8bd6-72426127d0a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:09.144470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce220b-a37a-4ebf-8bd6-72426127d0a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:09.146214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce220b-a37a-4ebf-8bd6-72426127d0a4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:09.170945 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1231/2464] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:55:09.177832 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fb12c-5fb9-4f04-a432-edb48f1ce52f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:09.191862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fb12c-5fb9-4f04-a432-edb48f1ce52f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:09.197418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb86f0-9adb-4a88-8553-b986eb3ace30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:09.198039 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb86f0-9adb-4a88-8553-b986eb3ace30 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:09.198492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb86f0-9adb-4a88-8553-b986eb3ace30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:09.198768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb86f0-9adb-4a88-8553-b986eb3ace30 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:09.199455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb86f0-9adb-4a88-8553-b986eb3ace30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:09.201415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb86f0-9adb-4a88-8553-b986eb3ace30 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:09.243573 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1234/2465] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:09 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:09.246652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce220b-a37a-4ebf-8bd6-72426127d0a4 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:09.261095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce220b-a37a-4ebf-8bd6-72426127d0a4 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:09.340834 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac03e5a-7481-4ec0-9a89-dbdd4f9427cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:09.341640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac03e5a-7481-4ec0-9a89-dbdd4f9427cd admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:09.342028 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac03e5a-7481-4ec0-9a89-dbdd4f9427cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:09.342401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac03e5a-7481-4ec0-9a89-dbdd4f9427cd admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:09.343168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac03e5a-7481-4ec0-9a89-dbdd4f9427cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:09.345368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac03e5a-7481-4ec0-9a89-dbdd4f9427cd admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:09.373921 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1232/2466] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:09 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:09.375264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb86f0-9adb-4a88-8553-b986eb3ace30 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:09.389270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb86f0-9adb-4a88-8553-b986eb3ace30 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:09.394483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253fd74-19fe-4e67-93db-fc3c98a516f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:09.395105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253fd74-19fe-4e67-93db-fc3c98a516f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:09.395419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253fd74-19fe-4e67-93db-fc3c98a516f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:09.395696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253fd74-19fe-4e67-93db-fc3c98a516f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:09.396342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253fd74-19fe-4e67-93db-fc3c98a516f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:09.398600 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253fd74-19fe-4e67-93db-fc3c98a516f6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:09.412948 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1235/2467] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:09.417541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac03e5a-7481-4ec0-9a89-dbdd4f9427cd admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:09.429041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac03e5a-7481-4ec0-9a89-dbdd4f9427cd admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:09.434314 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f645dbe-f72f-46b8-bfc3-e5ec4cd6c6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:09.435004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f645dbe-f72f-46b8-bfc3-e5ec4cd6c6ad admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:09.435383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f645dbe-f72f-46b8-bfc3-e5ec4cd6c6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:09.435723 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f645dbe-f72f-46b8-bfc3-e5ec4cd6c6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:09.436465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f645dbe-f72f-46b8-bfc3-e5ec4cd6c6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:09.438150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f645dbe-f72f-46b8-bfc3-e5ec4cd6c6ad admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:09.453818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1233/2468] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:09.457994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253fd74-19fe-4e67-93db-fc3c98a516f6 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:09.470089 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253fd74-19fe-4e67-93db-fc3c98a516f6 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:09.475706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cb531-b34a-488a-87f5-baf845dba1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:09.476644 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cb531-b34a-488a-87f5-baf845dba1fa admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:09.476985 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cb531-b34a-488a-87f5-baf845dba1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:09.477266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cb531-b34a-488a-87f5-baf845dba1fa admin admin] PATH_INFO: `/v3/projects/e007477ac9c144c186dba701bb5a1841/users/469bc7aa7e3f43f1bd88ac1870b48710/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:09.494875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cb531-b34a-488a-87f5-baf845dba1fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e007477ac9c144c186dba701bb5a1841, user_id=469bc7aa7e3f43f1bd88ac1870b48710, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:09.497893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cb531-b34a-488a-87f5-baf845dba1fa admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:09.515987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1236/2469] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:09 2022] PUT /identity/v3/projects/e007477ac9c144c186dba701bb5a1841/users/469bc7aa7e3f43f1bd88ac1870b48710/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:09.528206 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b2856-3b84-4fac-8e80-e656fbe8d2cd None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:09.528943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b2856-3b84-4fac-8e80-e656fbe8d2cd None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:09.528943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b2856-3b84-4fac-8e80-e656fbe8d2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:09.560959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-3a8b2856-3b84-4fac-8e80-e656fbe8d2cd None None] MFA Rules not processed for user `469bc7aa7e3f43f1bd88ac1870b48710`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:09.562207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3a8b2856-3b84-4fac-8e80-e656fbe8d2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:09.567827 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:09.645023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1234/2470] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:09.659074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cb531-b34a-488a-87f5-baf845dba1fa admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:09.660935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3c8cb531-b34a-488a-87f5-baf845dba1fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:09.743794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cb531-b34a-488a-87f5-baf845dba1fa admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:09.747948 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bd637-2614-4f41-95c7-c2e71106063a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:09.748835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bd637-2614-4f41-95c7-c2e71106063a service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:09.749342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bd637-2614-4f41-95c7-c2e71106063a service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:09.749829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bd637-2614-4f41-95c7-c2e71106063a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:09.760321 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3bd637-2614-4f41-95c7-c2e71106063a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:09.761954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3bd637-2614-4f41-95c7-c2e71106063a service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:09.771938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1237/2471] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:19.641636 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=24 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:19.767505 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:20.325444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc5904-29de-4e56-a146-b6c60b4a38bb None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:20.325737 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc5904-29de-4e56-a146-b6c60b4a38bb None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:20.326124 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc5904-29de-4e56-a146-b6c60b4a38bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:20.346057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-05bc5904-29de-4e56-a146-b6c60b4a38bb None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:20.347327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-05bc5904-29de-4e56-a146-b6c60b4a38bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:20.420251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1235/2472] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:55:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:20.433380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bd637-2614-4f41-95c7-c2e71106063a service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:20.435635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9d3bd637-2614-4f41-95c7-c2e71106063a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:20.522375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bd637-2614-4f41-95c7-c2e71106063a service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:20.527825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e5cdf-51d7-4e39-ba25-9993062d93de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:20.528432 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e5cdf-51d7-4e39-ba25-9993062d93de service swift] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:20.528710 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e5cdf-51d7-4e39-ba25-9993062d93de service swift] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:20.528978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e5cdf-51d7-4e39-ba25-9993062d93de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:20.537443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e5cdf-51d7-4e39-ba25-9993062d93de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:20.540231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e5cdf-51d7-4e39-ba25-9993062d93de service swift] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:20.549074 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1238/2473] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:55:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:55:21.511866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc5904-29de-4e56-a146-b6c60b4a38bb None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:21.624984 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc5904-29de-4e56-a146-b6c60b4a38bb None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:21.631491 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:21.632255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:21.632535 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:21.632808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] PATH_INFO: `/v3/users/469bc7aa7e3f43f1bd88ac1870b48710` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:21.634439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=469bc7aa7e3f43f1bd88ac1870b48710)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:21.637121 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:21.708485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '469bc7aa7e3f43f1bd88ac1870b48710'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:21.727023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '469bc7aa7e3f43f1bd88ac1870b48710'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:21.728354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '469bc7aa7e3f43f1bd88ac1870b48710'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:21.729346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '469bc7aa7e3f43f1bd88ac1870b48710'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:21.744055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '469bc7aa7e3f43f1bd88ac1870b48710'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:21.766746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1236/2474] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:21 2022] DELETE /identity/v3/users/469bc7aa7e3f43f1bd88ac1870b48710 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:21.774370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1face28f-5778-40b4-a0ac-bd8149066bd4 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:21.775191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1face28f-5778-40b4-a0ac-bd8149066bd4 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:21.775191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1face28f-5778-40b4-a0ac-bd8149066bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:21.798744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-1face28f-5778-40b4-a0ac-bd8149066bd4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:21.800341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1face28f-5778-40b4-a0ac-bd8149066bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:21.889477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1239/2475] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:21.900296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:21.908072 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:21.957893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:22.000879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb59b8-e6cd-4a31-a239-89d9104d3b01 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:22.007539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f483b86-d354-416d-8d53-83cd4b70b0a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:22.008440 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f483b86-d354-416d-8d53-83cd4b70b0a5 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:22.008893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f483b86-d354-416d-8d53-83cd4b70b0a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:22.009566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f483b86-d354-416d-8d53-83cd4b70b0a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:22.025991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f483b86-d354-416d-8d53-83cd4b70b0a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:22.027742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f483b86-d354-416d-8d53-83cd4b70b0a5 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:22.041412 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1237/2476] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:22.624246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1face28f-5778-40b4-a0ac-bd8149066bd4 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:22.625951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1face28f-5778-40b4-a0ac-bd8149066bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:22.647273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1face28f-5778-40b4-a0ac-bd8149066bd4 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:22.653940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4154d-d497-464c-b76b-2e08ed512344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:22.654755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4154d-d497-464c-b76b-2e08ed512344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:22.655225 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4154d-d497-464c-b76b-2e08ed512344 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:22.655410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4154d-d497-464c-b76b-2e08ed512344 admin admin] PATH_INFO: `/v3/projects/e007477ac9c144c186dba701bb5a1841` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:22.656912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4154d-d497-464c-b76b-2e08ed512344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e007477ac9c144c186dba701bb5a1841)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:22.659430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4154d-d497-464c-b76b-2e08ed512344 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:22.745348 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68c4154d-d497-464c-b76b-2e08ed512344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e007477ac9c144c186dba701bb5a1841', 'request_id': 'req-68c4154d-d497-464c-b76b-2e08ed512344'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:22.753310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68c4154d-d497-464c-b76b-2e08ed512344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e007477ac9c144c186dba701bb5a1841', 'request_id': 'req-68c4154d-d497-464c-b76b-2e08ed512344'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:22.761162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68c4154d-d497-464c-b76b-2e08ed512344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e007477ac9c144c186dba701bb5a1841', 'request_id': 'req-68c4154d-d497-464c-b76b-2e08ed512344'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:22.778806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68c4154d-d497-464c-b76b-2e08ed512344 admin admin] The token cache is being invalidate because project e007477ac9c144c186dba701bb5a1841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:22.779502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68c4154d-d497-464c-b76b-2e08ed512344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:22.780007 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-68c4154d-d497-464c-b76b-2e08ed512344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:22.782065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1240/2477] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:22 2022] DELETE /identity/v3/projects/e007477ac9c144c186dba701bb5a1841 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:22.787048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd19b4-5f6e-46af-83c7-645a80041c1b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:22.787607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd19b4-5f6e-46af-83c7-645a80041c1b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:22.787821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd19b4-5f6e-46af-83c7-645a80041c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:22.818593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b5cd19b4-5f6e-46af-83c7-645a80041c1b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:22.820779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b5cd19b4-5f6e-46af-83c7-645a80041c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:22.918253 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1238/2478] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:22.928389 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f13d2e-577f-45ca-8134-75bf60e64fe7 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:22.930015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f13d2e-577f-45ca-8134-75bf60e64fe7 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:22.930284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f13d2e-577f-45ca-8134-75bf60e64fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:22.953763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-78f13d2e-577f-45ca-8134-75bf60e64fe7 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:22.955018 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-78f13d2e-577f-45ca-8134-75bf60e64fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:22.966377 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1241/2479] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:22.971984 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd19b4-5f6e-46af-83c7-645a80041c1b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:22.988705 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd19b4-5f6e-46af-83c7-645a80041c1b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:22.999415 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fba6d9-fc46-4c5e-abb6-88e04993ac08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:23.000345 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fba6d9-fc46-4c5e-abb6-88e04993ac08 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:23.000789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fba6d9-fc46-4c5e-abb6-88e04993ac08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:23.001281 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fba6d9-fc46-4c5e-abb6-88e04993ac08 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:23.002600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fba6d9-fc46-4c5e-abb6-88e04993ac08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:23.003214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fba6d9-fc46-4c5e-abb6-88e04993ac08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:23.006043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fba6d9-fc46-4c5e-abb6-88e04993ac08 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:23.024898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1239/2480] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:55:23.032778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f13d2e-577f-45ca-8134-75bf60e64fe7 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:23.055063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f13d2e-577f-45ca-8134-75bf60e64fe7 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:23.063363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e00a17-4947-4512-b0db-c262bdb05e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:23.064286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e00a17-4947-4512-b0db-c262bdb05e0d admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:23.064833 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e00a17-4947-4512-b0db-c262bdb05e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:23.065146 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e00a17-4947-4512-b0db-c262bdb05e0d admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:23.065938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e00a17-4947-4512-b0db-c262bdb05e0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:23.068298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e00a17-4947-4512-b0db-c262bdb05e0d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:23.075051 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:23.136856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1242/2481] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:23 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:23.139321 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fba6d9-fc46-4c5e-abb6-88e04993ac08 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:23.155017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fba6d9-fc46-4c5e-abb6-88e04993ac08 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:23.297941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f5a66-f48f-4260-b215-759243731ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:23.299453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f5a66-f48f-4260-b215-759243731ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:23.300151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f5a66-f48f-4260-b215-759243731ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:23.300656 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f5a66-f48f-4260-b215-759243731ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:23.301886 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f5a66-f48f-4260-b215-759243731ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:23.306103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f5a66-f48f-4260-b215-759243731ad2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:23.353462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1240/2482] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:23 2022] POST /identity/v3/users => generated 282 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:23.358258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e00a17-4947-4512-b0db-c262bdb05e0d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:23.370713 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e00a17-4947-4512-b0db-c262bdb05e0d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:23.377723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7396fa8-b264-47b6-8b5d-efaeb6a94f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:23.378297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7396fa8-b264-47b6-8b5d-efaeb6a94f14 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:23.378681 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7396fa8-b264-47b6-8b5d-efaeb6a94f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:23.379061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7396fa8-b264-47b6-8b5d-efaeb6a94f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:23.379892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7396fa8-b264-47b6-8b5d-efaeb6a94f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:23.381801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7396fa8-b264-47b6-8b5d-efaeb6a94f14 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:23.398327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1243/2483] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:23.400583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f5a66-f48f-4260-b215-759243731ad2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:23.422349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f5a66-f48f-4260-b215-759243731ad2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:23.430796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b1ec9-3ad5-42a6-ba16-5d2ceb2d7548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:23.431768 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b1ec9-3ad5-42a6-ba16-5d2ceb2d7548 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:23.432092 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b1ec9-3ad5-42a6-ba16-5d2ceb2d7548 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:23.432363 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b1ec9-3ad5-42a6-ba16-5d2ceb2d7548 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:23.433071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b1ec9-3ad5-42a6-ba16-5d2ceb2d7548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:23.446651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b1ec9-3ad5-42a6-ba16-5d2ceb2d7548 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:23.477257 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7396fa8-b264-47b6-8b5d-efaeb6a94f14 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:23.478470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1241/2484] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:23.492120 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7396fa8-b264-47b6-8b5d-efaeb6a94f14 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:23.497453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d43e67-ad07-48bf-b1dd-ed60ab0b1d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:23.498139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d43e67-ad07-48bf-b1dd-ed60ab0b1d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:23.498452 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d43e67-ad07-48bf-b1dd-ed60ab0b1d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:23.498815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d43e67-ad07-48bf-b1dd-ed60ab0b1d01 admin admin] PATH_INFO: `/v3/projects/b21b175b64694a469b40c08657d996af/users/2d743f4dba654ce4b536c9231b82fb5d/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:23.546960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d43e67-ad07-48bf-b1dd-ed60ab0b1d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21b175b64694a469b40c08657d996af, user_id=2d743f4dba654ce4b536c9231b82fb5d, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:23.565751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d43e67-ad07-48bf-b1dd-ed60ab0b1d01 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:23.647161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1244/2485] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:23 2022] PUT /identity/v3/projects/b21b175b64694a469b40c08657d996af/users/2d743f4dba654ce4b536c9231b82fb5d/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:55:23.659693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82d3ea-28c6-43df-adc0-907fcd083180 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:23.660078 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82d3ea-28c6-43df-adc0-907fcd083180 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:23.660186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82d3ea-28c6-43df-adc0-907fcd083180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:23.737962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-fb82d3ea-28c6-43df-adc0-907fcd083180 None None] MFA Rules not processed for user `2d743f4dba654ce4b536c9231b82fb5d`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:23.739430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-fb82d3ea-28c6-43df-adc0-907fcd083180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:23.797238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:23.886791 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1242/2486] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:23.892080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d43e67-ad07-48bf-b1dd-ed60ab0b1d01 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:24.006072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d43e67-ad07-48bf-b1dd-ed60ab0b1d01 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:24.044897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c126e8b-d2a8-47ab-a270-d7e6cefa337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:24.045524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126e8b-d2a8-47ab-a270-d7e6cefa337a admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:24.045803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126e8b-d2a8-47ab-a270-d7e6cefa337a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:24.046076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126e8b-d2a8-47ab-a270-d7e6cefa337a admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:24.047280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c126e8b-d2a8-47ab-a270-d7e6cefa337a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:24.054762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c126e8b-d2a8-47ab-a270-d7e6cefa337a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:24.128448 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1245/2487] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:23 2022] POST /identity/v3/projects => generated 363 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:24.141821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82d3ea-28c6-43df-adc0-907fcd083180 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:24.183793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82d3ea-28c6-43df-adc0-907fcd083180 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:24.386760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f5459-d492-47aa-bc24-2f145201e5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:24.387440 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f5459-d492-47aa-bc24-2f145201e5e4 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:24.387724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f5459-d492-47aa-bc24-2f145201e5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:24.387996 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f5459-d492-47aa-bc24-2f145201e5e4 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:24.388715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f5459-d492-47aa-bc24-2f145201e5e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:24.391022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f5459-d492-47aa-bc24-2f145201e5e4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:24.450497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1243/2488] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:24 2022] POST /identity/v3/users => generated 288 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:24.454047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c126e8b-d2a8-47ab-a270-d7e6cefa337a admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:24.476623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c126e8b-d2a8-47ab-a270-d7e6cefa337a admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:24.487168 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951a318-f38f-4cde-a69a-eeba4cdce434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:24.488075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951a318-f38f-4cde-a69a-eeba4cdce434 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:24.488555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951a318-f38f-4cde-a69a-eeba4cdce434 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:24.489018 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951a318-f38f-4cde-a69a-eeba4cdce434 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:24.490245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951a318-f38f-4cde-a69a-eeba4cdce434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:24.496734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951a318-f38f-4cde-a69a-eeba4cdce434 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:24.531786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1246/2489] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:24.540889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f5459-d492-47aa-bc24-2f145201e5e4 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:24.573764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f5459-d492-47aa-bc24-2f145201e5e4 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:24.579946 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e79c4f-944a-477c-a1e1-124ba693a3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:24.581060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e79c4f-944a-477c-a1e1-124ba693a3e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:24.582775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e79c4f-944a-477c-a1e1-124ba693a3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:24.582775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e79c4f-944a-477c-a1e1-124ba693a3e5 admin admin] PATH_INFO: `/v3/projects/ff1d9a4264b4406e9eff97fc68cc89fd/users/31c5c5f407cf434f865fce67b3450d9f/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:24.616732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e79c4f-944a-477c-a1e1-124ba693a3e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1d9a4264b4406e9eff97fc68cc89fd, user_id=31c5c5f407cf434f865fce67b3450d9f, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:24.623585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e79c4f-944a-477c-a1e1-124ba693a3e5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:24.656335 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951a318-f38f-4cde-a69a-eeba4cdce434 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:24.665714 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1244/2490] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:24 2022] PUT /identity/v3/projects/ff1d9a4264b4406e9eff97fc68cc89fd/users/31c5c5f407cf434f865fce67b3450d9f/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:24.677831 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951a318-f38f-4cde-a69a-eeba4cdce434 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:24.808896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d32dc-6e80-4059-b9c0-cf34e99ad910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:24.809897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d32dc-6e80-4059-b9c0-cf34e99ad910 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:24.810192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d32dc-6e80-4059-b9c0-cf34e99ad910 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:24.810594 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d32dc-6e80-4059-b9c0-cf34e99ad910 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:24.811298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d32dc-6e80-4059-b9c0-cf34e99ad910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:24.813140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d32dc-6e80-4059-b9c0-cf34e99ad910 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:24.828622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1247/2491] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:24.833389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e79c4f-944a-477c-a1e1-124ba693a3e5 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:24.849128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e79c4f-944a-477c-a1e1-124ba693a3e5 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:24.856533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2b1ff-1d9e-4938-a618-5d22e502f65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:24.857784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2b1ff-1d9e-4938-a618-5d22e502f65f admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:24.858151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2b1ff-1d9e-4938-a618-5d22e502f65f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:24.858434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2b1ff-1d9e-4938-a618-5d22e502f65f admin admin] PATH_INFO: `/v3/projects/ff1d9a4264b4406e9eff97fc68cc89fd/users/31c5c5f407cf434f865fce67b3450d9f/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:24.860955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2b1ff-1d9e-4938-a618-5d22e502f65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1d9a4264b4406e9eff97fc68cc89fd, user_id=31c5c5f407cf434f865fce67b3450d9f, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:24.864875 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2b1ff-1d9e-4938-a618-5d22e502f65f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:24.885443 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1245/2492] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:24 2022] PUT /identity/v3/projects/ff1d9a4264b4406e9eff97fc68cc89fd/users/31c5c5f407cf434f865fce67b3450d9f/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:55:24.904148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be7d43-8d24-4123-a2f7-c5b0086768be None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:24.905206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be7d43-8d24-4123-a2f7-c5b0086768be None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:24.905558 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be7d43-8d24-4123-a2f7-c5b0086768be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:24.958862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b8be7d43-8d24-4123-a2f7-c5b0086768be None None] MFA Rules not processed for user `31c5c5f407cf434f865fce67b3450d9f`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:24.964841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b8be7d43-8d24-4123-a2f7-c5b0086768be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:24.983647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:25.119403 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1248/2493] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:25.134136 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2b1ff-1d9e-4938-a618-5d22e502f65f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:25.137641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c8c2b1ff-1d9e-4938-a618-5d22e502f65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:25.250110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2b1ff-1d9e-4938-a618-5d22e502f65f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:25.258228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091f2fb-6a21-48d7-bb5a-24b5d3e8ee47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:25.259171 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091f2fb-6a21-48d7-bb5a-24b5d3e8ee47 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:25.259629 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091f2fb-6a21-48d7-bb5a-24b5d3e8ee47 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:25.259949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091f2fb-6a21-48d7-bb5a-24b5d3e8ee47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:25.273894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091f2fb-6a21-48d7-bb5a-24b5d3e8ee47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:25.277401 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091f2fb-6a21-48d7-bb5a-24b5d3e8ee47 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:25.297428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1246/2494] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:25.437414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be7d43-8d24-4123-a2f7-c5b0086768be None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:25.451004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be7d43-8d24-4123-a2f7-c5b0086768be None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:25.456263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a66ce9-f0a8-41cb-b2ca-b396ea2b02af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:25.456965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a66ce9-f0a8-41cb-b2ca-b396ea2b02af service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:25.457337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a66ce9-f0a8-41cb-b2ca-b396ea2b02af service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:25.457649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a66ce9-f0a8-41cb-b2ca-b396ea2b02af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:25.558876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a66ce9-f0a8-41cb-b2ca-b396ea2b02af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:25.560920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a66ce9-f0a8-41cb-b2ca-b396ea2b02af service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:25.578585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1249/2495] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:33.707693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335187d2-310f-4fa3-82fb-e7d969f0c942 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:33.707693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335187d2-310f-4fa3-82fb-e7d969f0c942 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:33.707693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335187d2-310f-4fa3-82fb-e7d969f0c942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:33.740325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-335187d2-310f-4fa3-82fb-e7d969f0c942 None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:33.742756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-335187d2-310f-4fa3-82fb-e7d969f0c942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:33.768483 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:33.838794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1247/2496] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:55:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:33.869043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a66ce9-f0a8-41cb-b2ca-b396ea2b02af service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:33.870534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b2a66ce9-f0a8-41cb-b2ca-b396ea2b02af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:33.970155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a66ce9-f0a8-41cb-b2ca-b396ea2b02af service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:33.981802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b48bb-97e7-42a5-add5-927db7c38657 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:33.983440 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b48bb-97e7-42a5-add5-927db7c38657 service swift] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:33.984454 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b48bb-97e7-42a5-add5-927db7c38657 service swift] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:33.985004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b48bb-97e7-42a5-add5-927db7c38657 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:34.004197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b48bb-97e7-42a5-add5-927db7c38657 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:34.011159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b48bb-97e7-42a5-add5-927db7c38657 service swift] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:34.026158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1250/2497] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:55:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:55:37.650712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335187d2-310f-4fa3-82fb-e7d969f0c942 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:37.751164 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335187d2-310f-4fa3-82fb-e7d969f0c942 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:37.758713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:37.759528 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:37.759912 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:37.760520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] PATH_INFO: `/v3/users/2d743f4dba654ce4b536c9231b82fb5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:37.761792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d743f4dba654ce4b536c9231b82fb5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:37.764182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:37.913067 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d743f4dba654ce4b536c9231b82fb5d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:37.974719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d743f4dba654ce4b536c9231b82fb5d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:37.979811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d743f4dba654ce4b536c9231b82fb5d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:37.982967 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d743f4dba654ce4b536c9231b82fb5d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:38.022195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d743f4dba654ce4b536c9231b82fb5d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:38.103946 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1248/2498] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:37 2022] DELETE /identity/v3/users/2d743f4dba654ce4b536c9231b82fb5d => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:55:38.108400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea21c79-b129-49fe-82bb-27935101dea4 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:38.109360 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea21c79-b129-49fe-82bb-27935101dea4 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:38.109360 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea21c79-b129-49fe-82bb-27935101dea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:38.145307 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-fea21c79-b129-49fe-82bb-27935101dea4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:38.147851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fea21c79-b129-49fe-82bb-27935101dea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:38.208654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:38.273536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1251/2499] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:38.285730 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:38.288204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:38.384811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:38.419753 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17825c-fd8b-4dd0-a46b-58726f5f5f33 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:38.431316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476e62a-071f-4530-9377-ab8598c66e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:38.432153 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476e62a-071f-4530-9377-ab8598c66e95 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:38.433105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476e62a-071f-4530-9377-ab8598c66e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:38.433491 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476e62a-071f-4530-9377-ab8598c66e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:38.454228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476e62a-071f-4530-9377-ab8598c66e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:38.456280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476e62a-071f-4530-9377-ab8598c66e95 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:38.469626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1249/2500] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:39.325306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea21c79-b129-49fe-82bb-27935101dea4 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:39.330559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fea21c79-b129-49fe-82bb-27935101dea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:39.383740 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea21c79-b129-49fe-82bb-27935101dea4 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:39.391433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:39.393463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:39.394094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:39.394567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] PATH_INFO: `/v3/projects/b21b175b64694a469b40c08657d996af` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:39.396300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21b175b64694a469b40c08657d996af)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:39.399708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:39.484753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21b175b64694a469b40c08657d996af', 'request_id': 'req-8969f195-b823-4723-9293-81e4baf78c9c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:39.511873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21b175b64694a469b40c08657d996af', 'request_id': 'req-8969f195-b823-4723-9293-81e4baf78c9c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:39.519738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21b175b64694a469b40c08657d996af', 'request_id': 'req-8969f195-b823-4723-9293-81e4baf78c9c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:39.539962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] The token cache is being invalidate because project b21b175b64694a469b40c08657d996af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:39.540427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:39.541430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:39.547739 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476e62a-071f-4530-9377-ab8598c66e95 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:39.548659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1252/2501] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:39 2022] DELETE /identity/v3/projects/b21b175b64694a469b40c08657d996af => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:39.560087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476e62a-071f-4530-9377-ab8598c66e95 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:39.633835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:39.634640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:39.635512 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:39.635512 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] PATH_INFO: `/v3/users/31c5c5f407cf434f865fce67b3450d9f` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:39.636657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c5c5f407cf434f865fce67b3450d9f)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:39.638904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:39.743865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c5c5f407cf434f865fce67b3450d9f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:39.775033 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c5c5f407cf434f865fce67b3450d9f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:39.775755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c5c5f407cf434f865fce67b3450d9f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:39.776459 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c5c5f407cf434f865fce67b3450d9f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:39.790168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c5c5f407cf434f865fce67b3450d9f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:39.825529 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1250/2502] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:39 2022] DELETE /identity/v3/users/31c5c5f407cf434f865fce67b3450d9f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:40.470917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:40.473099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:40.498558 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969f195-b823-4723-9293-81e4baf78c9c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:40.636126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:40.637854 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:40.711222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c96ddb-e62f-4493-a119-72d1d7043a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:40.712365 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c96ddb-e62f-4493-a119-72d1d7043a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:40.713258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c96ddb-e62f-4493-a119-72d1d7043a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:40.715273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c96ddb-e62f-4493-a119-72d1d7043a40 admin admin] PATH_INFO: `/v3/projects/ff1d9a4264b4406e9eff97fc68cc89fd` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:40.718756 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c96ddb-e62f-4493-a119-72d1d7043a40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff1d9a4264b4406e9eff97fc68cc89fd)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:40.724738 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c96ddb-e62f-4493-a119-72d1d7043a40 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:40.789933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0206d70-5c39-4de2-b5af-cd93d9c9a55c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:40.798861 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a0c44-8829-42d3-bbdc-f7d3be26fe11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:40.799792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a0c44-8829-42d3-bbdc-f7d3be26fe11 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:40.800296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a0c44-8829-42d3-bbdc-f7d3be26fe11 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:40.800818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a0c44-8829-42d3-bbdc-f7d3be26fe11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:40.804114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-986a0c44-8829-42d3-bbdc-f7d3be26fe11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:40.823757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a0c44-8829-42d3-bbdc-f7d3be26fe11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:40.837089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a0c44-8829-42d3-bbdc-f7d3be26fe11 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:40.857829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-09c96ddb-e62f-4493-a119-72d1d7043a40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff1d9a4264b4406e9eff97fc68cc89fd', 'request_id': 'req-09c96ddb-e62f-4493-a119-72d1d7043a40'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:40.862385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1251/2503] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:55:40.869885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-09c96ddb-e62f-4493-a119-72d1d7043a40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff1d9a4264b4406e9eff97fc68cc89fd', 'request_id': 'req-09c96ddb-e62f-4493-a119-72d1d7043a40'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:40.875561 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:40.879245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-09c96ddb-e62f-4493-a119-72d1d7043a40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff1d9a4264b4406e9eff97fc68cc89fd', 'request_id': 'req-09c96ddb-e62f-4493-a119-72d1d7043a40'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:40.901231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-09c96ddb-e62f-4493-a119-72d1d7043a40 admin admin] The token cache is being invalidate because project ff1d9a4264b4406e9eff97fc68cc89fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:40.901789 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-09c96ddb-e62f-4493-a119-72d1d7043a40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:40.903423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-09c96ddb-e62f-4493-a119-72d1d7043a40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:40.906533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1253/2504] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:40 2022] DELETE /identity/v3/projects/ff1d9a4264b4406e9eff97fc68cc89fd => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:40.911242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f2fd-583a-45f9-8350-5c1a93d434ff None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:40.911611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f2fd-583a-45f9-8350-5c1a93d434ff None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:40.912033 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f2fd-583a-45f9-8350-5c1a93d434ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:40.937219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:40.954961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-dd74f2fd-583a-45f9-8350-5c1a93d434ff None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:40.959949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-dd74f2fd-583a-45f9-8350-5c1a93d434ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:41.120306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1252/2505] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:41.128952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960bab2-e2dd-4235-ba5b-6fab2d1cb8bb None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:41.129675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960bab2-e2dd-4235-ba5b-6fab2d1cb8bb None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:41.130256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960bab2-e2dd-4235-ba5b-6fab2d1cb8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:41.172249 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4960bab2-e2dd-4235-ba5b-6fab2d1cb8bb None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:41.173653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4960bab2-e2dd-4235-ba5b-6fab2d1cb8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:41.186409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1254/2506] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:41.190982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74f2fd-583a-45f9-8350-5c1a93d434ff None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:41.202042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74f2fd-583a-45f9-8350-5c1a93d434ff None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:41.206880 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da7009-f47d-49d9-bbd4-73c1073c8d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:41.207555 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da7009-f47d-49d9-bbd4-73c1073c8d80 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:41.207837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da7009-f47d-49d9-bbd4-73c1073c8d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:41.208252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da7009-f47d-49d9-bbd4-73c1073c8d80 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:41.209113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da7009-f47d-49d9-bbd4-73c1073c8d80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:41.210032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da7009-f47d-49d9-bbd4-73c1073c8d80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:41.211709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da7009-f47d-49d9-bbd4-73c1073c8d80 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:41.226065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1253/2507] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:41.231042 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960bab2-e2dd-4235-ba5b-6fab2d1cb8bb None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:41.243492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960bab2-e2dd-4235-ba5b-6fab2d1cb8bb None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:41.255989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc031cc-a907-4159-90ea-3be60cc858d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:41.257042 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc031cc-a907-4159-90ea-3be60cc858d6 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:41.257724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc031cc-a907-4159-90ea-3be60cc858d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:41.258220 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc031cc-a907-4159-90ea-3be60cc858d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:41.259009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc031cc-a907-4159-90ea-3be60cc858d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:41.261346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc031cc-a907-4159-90ea-3be60cc858d6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:41.299903 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1255/2508] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:41 2022] POST /identity/v3/projects => generated 367 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:41.304490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da7009-f47d-49d9-bbd4-73c1073c8d80 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:41.319986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da7009-f47d-49d9-bbd4-73c1073c8d80 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:41.458851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399ba03-d789-43bb-8849-d8311e61bf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:41.460210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ba03-d789-43bb-8849-d8311e61bf7e admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:41.460905 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ba03-d789-43bb-8849-d8311e61bf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:41.461188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ba03-d789-43bb-8849-d8311e61bf7e admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:41.461846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399ba03-d789-43bb-8849-d8311e61bf7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:41.464041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399ba03-d789-43bb-8849-d8311e61bf7e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:41.511056 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1254/2509] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:41 2022] POST /identity/v3/users => generated 284 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:41.512169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc031cc-a907-4159-90ea-3be60cc858d6 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:41.525875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc031cc-a907-4159-90ea-3be60cc858d6 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:41.544393 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317e30-da24-4ee6-966f-0df362508644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:41.544858 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317e30-da24-4ee6-966f-0df362508644 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:41.545386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317e30-da24-4ee6-966f-0df362508644 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:41.545725 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317e30-da24-4ee6-966f-0df362508644 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:41.546563 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13317e30-da24-4ee6-966f-0df362508644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:41.548813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13317e30-da24-4ee6-966f-0df362508644 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:41.588465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1256/2510] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:55:41.591072 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399ba03-d789-43bb-8849-d8311e61bf7e admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:41.608332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399ba03-d789-43bb-8849-d8311e61bf7e admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:41.615273 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f865c-2282-4078-9fc7-309b1cb62d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:41.619659 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f865c-2282-4078-9fc7-309b1cb62d2b admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:41.621138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f865c-2282-4078-9fc7-309b1cb62d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:41.621685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f865c-2282-4078-9fc7-309b1cb62d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:41.622852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f865c-2282-4078-9fc7-309b1cb62d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:41.624842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f865c-2282-4078-9fc7-309b1cb62d2b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:41.645157 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1255/2511] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:41.645819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317e30-da24-4ee6-966f-0df362508644 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:41.656520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317e30-da24-4ee6-966f-0df362508644 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:41.663540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67108074-eb01-4382-8fad-4a11fc19fdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:41.663866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67108074-eb01-4382-8fad-4a11fc19fdd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:41.664149 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67108074-eb01-4382-8fad-4a11fc19fdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:41.664420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67108074-eb01-4382-8fad-4a11fc19fdd4 admin admin] PATH_INFO: `/v3/projects/829529842e444ba8aef3aa5b75832ba6/users/5ef138e7ba8441e4ac98e193737cfa75/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:41.684498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67108074-eb01-4382-8fad-4a11fc19fdd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=829529842e444ba8aef3aa5b75832ba6, user_id=5ef138e7ba8441e4ac98e193737cfa75, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:41.687191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67108074-eb01-4382-8fad-4a11fc19fdd4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:41.716742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1257/2512] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:41 2022] PUT /identity/v3/projects/829529842e444ba8aef3aa5b75832ba6/users/5ef138e7ba8441e4ac98e193737cfa75/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:55:41.719962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ad265-6739-4e9d-b9bb-68d99b774ddb None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:41.720527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ad265-6739-4e9d-b9bb-68d99b774ddb None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:41.721174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ad265-6739-4e9d-b9bb-68d99b774ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:41.797691 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-774ad265-6739-4e9d-b9bb-68d99b774ddb None None] MFA Rules not processed for user `5ef138e7ba8441e4ac98e193737cfa75`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:41.799755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-774ad265-6739-4e9d-b9bb-68d99b774ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:41.950178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1256/2513] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:41.952904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67108074-eb01-4382-8fad-4a11fc19fdd4 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:42.016386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b93c9d-6803-4ae6-925e-e8aaf478d2ad None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:42.017372 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b93c9d-6803-4ae6-925e-e8aaf478d2ad None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:42.017954 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b93c9d-6803-4ae6-925e-e8aaf478d2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:42.032671 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:42.060327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-80b93c9d-6803-4ae6-925e-e8aaf478d2ad None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:42.061593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-80b93c9d-6803-4ae6-925e-e8aaf478d2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:42.194741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67108074-eb01-4382-8fad-4a11fc19fdd4 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:42.213401 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa601d35-f362-4851-950e-e29988f94cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:42.214037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa601d35-f362-4851-950e-e29988f94cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:42.214315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa601d35-f362-4851-950e-e29988f94cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:42.214982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa601d35-f362-4851-950e-e29988f94cf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:42.216715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa601d35-f362-4851-950e-e29988f94cf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:42.226372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa601d35-f362-4851-950e-e29988f94cf0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:42.275620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1257/2514] 10.222.0.36 () {58 vars in 967 bytes} [Wed Aug 31 09:55:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:42.287597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b93c9d-6803-4ae6-925e-e8aaf478d2ad None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:42.290062 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-80b93c9d-6803-4ae6-925e-e8aaf478d2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:42.291598 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1258/2515] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:41 2022] POST /identity/v3/projects => generated 367 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:42.292776 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa601d35-f362-4851-950e-e29988f94cf0 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:42.430784 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa601d35-f362-4851-950e-e29988f94cf0 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:42.450288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c4654-ab37-44d8-ba2a-4278d450012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:42.451418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c4654-ab37-44d8-ba2a-4278d450012e admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:42.452017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c4654-ab37-44d8-ba2a-4278d450012e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:42.452144 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b93c9d-6803-4ae6-925e-e8aaf478d2ad None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:42.452653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c4654-ab37-44d8-ba2a-4278d450012e admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:42.453532 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c4654-ab37-44d8-ba2a-4278d450012e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:42.460469 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c4654-ab37-44d8-ba2a-4278d450012e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:42.505946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:DELETE=2,keystone:SELECT=91,keystone:INSERT=7 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:42.528304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1259/2516] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:42 2022] POST /identity/v3/users => generated 290 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:42.533018 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c4654-ab37-44d8-ba2a-4278d450012e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:42.556400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c4654-ab37-44d8-ba2a-4278d450012e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:42.571229 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830f605-0528-4e80-9151-3e1804cb9a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:42.572284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830f605-0528-4e80-9151-3e1804cb9a1c admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:42.572547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830f605-0528-4e80-9151-3e1804cb9a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:42.572929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830f605-0528-4e80-9151-3e1804cb9a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:42.573674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830f605-0528-4e80-9151-3e1804cb9a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:42.575916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830f605-0528-4e80-9151-3e1804cb9a1c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:42.607640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1260/2517] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:42.612834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830f605-0528-4e80-9151-3e1804cb9a1c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:42.639186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782ee28-9233-43fc-a771-4a85dc568693 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:42.639953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830f605-0528-4e80-9151-3e1804cb9a1c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:42.640371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782ee28-9233-43fc-a771-4a85dc568693 service swift] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:42.641188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782ee28-9233-43fc-a771-4a85dc568693 service swift] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:42.641691 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782ee28-9233-43fc-a771-4a85dc568693 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:42.654284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc812db8-ace4-4f09-b31a-2214f28f7f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:42.655139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc812db8-ace4-4f09-b31a-2214f28f7f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:42.655529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc812db8-ace4-4f09-b31a-2214f28f7f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:42.655908 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc812db8-ace4-4f09-b31a-2214f28f7f44 admin admin] PATH_INFO: `/v3/projects/1a5857905b30465f8337fde981b8e02e/users/1f98db6164ea4edf83aad5cadf7f3290/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:42.690333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc812db8-ace4-4f09-b31a-2214f28f7f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5857905b30465f8337fde981b8e02e, user_id=1f98db6164ea4edf83aad5cadf7f3290, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:42.691604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc812db8-ace4-4f09-b31a-2214f28f7f44 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:42.726771 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1261/2518] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:42 2022] PUT /identity/v3/projects/1a5857905b30465f8337fde981b8e02e/users/1f98db6164ea4edf83aad5cadf7f3290/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:42.730525 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc812db8-ace4-4f09-b31a-2214f28f7f44 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:42.739484 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:42.760360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782ee28-9233-43fc-a771-4a85dc568693 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:42.762137 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782ee28-9233-43fc-a771-4a85dc568693 service swift] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:42.775965 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1258/2519] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:55:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:55:42.909082 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc812db8-ace4-4f09-b31a-2214f28f7f44 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:42.916199 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1456b99-c8d6-4f08-849b-f85b5e0712b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:42.916914 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1456b99-c8d6-4f08-849b-f85b5e0712b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:42.917297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1456b99-c8d6-4f08-849b-f85b5e0712b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:42.917696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1456b99-c8d6-4f08-849b-f85b5e0712b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:42.918755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1456b99-c8d6-4f08-849b-f85b5e0712b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:42.920816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1456b99-c8d6-4f08-849b-f85b5e0712b7 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:42.935843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1262/2520] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:42.942753 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782ee28-9233-43fc-a771-4a85dc568693 service swift] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:42.982178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782ee28-9233-43fc-a771-4a85dc568693 service swift] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:42.987747 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20aed5-9919-4968-a802-e248dd593155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:42.990278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20aed5-9919-4968-a802-e248dd593155 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:42.992166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20aed5-9919-4968-a802-e248dd593155 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:42.992864 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20aed5-9919-4968-a802-e248dd593155 admin admin] PATH_INFO: `/v3/projects/1a5857905b30465f8337fde981b8e02e/users/1f98db6164ea4edf83aad5cadf7f3290/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:43.008039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20aed5-9919-4968-a802-e248dd593155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5857905b30465f8337fde981b8e02e, user_id=1f98db6164ea4edf83aad5cadf7f3290, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:43.013464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20aed5-9919-4968-a802-e248dd593155 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:43.050160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1259/2521] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:42 2022] PUT /identity/v3/projects/1a5857905b30465f8337fde981b8e02e/users/1f98db6164ea4edf83aad5cadf7f3290/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:43.064449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f50875-62b5-41ad-9c75-36015ddb5614 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:43.065950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f50875-62b5-41ad-9c75-36015ddb5614 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:43.066545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f50875-62b5-41ad-9c75-36015ddb5614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:43.114455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-c8f50875-62b5-41ad-9c75-36015ddb5614 None None] MFA Rules not processed for user `1f98db6164ea4edf83aad5cadf7f3290`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:43.117138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c8f50875-62b5-41ad-9c75-36015ddb5614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:43.254863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1263/2522] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:43.266715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20aed5-9919-4968-a802-e248dd593155 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:43.269199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ee20aed5-9919-4968-a802-e248dd593155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:43.397732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20aed5-9919-4968-a802-e248dd593155 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:43.407941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df5794-3b0b-47c3-a947-49b399816af9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:43.408908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df5794-3b0b-47c3-a947-49b399816af9 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:43.409390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df5794-3b0b-47c3-a947-49b399816af9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:43.410056 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df5794-3b0b-47c3-a947-49b399816af9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:43.428335 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df5794-3b0b-47c3-a947-49b399816af9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:43.430676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df5794-3b0b-47c3-a947-49b399816af9 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:43.462765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1260/2523] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:43.725532 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f50875-62b5-41ad-9c75-36015ddb5614 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:43.727270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c8f50875-62b5-41ad-9c75-36015ddb5614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:43.759549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f50875-62b5-41ad-9c75-36015ddb5614 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:43.777405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97134c7-6736-4dfc-bfda-035041671b2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:43.777405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97134c7-6736-4dfc-bfda-035041671b2b service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:43.777405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97134c7-6736-4dfc-bfda-035041671b2b service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:43.777405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97134c7-6736-4dfc-bfda-035041671b2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:43.914993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:43.964423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97134c7-6736-4dfc-bfda-035041671b2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:43.967882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97134c7-6736-4dfc-bfda-035041671b2b service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:43.987055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1264/2524] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:46.016190 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df5794-3b0b-47c3-a947-49b399816af9 service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:46.017709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-67df5794-3b0b-47c3-a947-49b399816af9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:46.129358 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97134c7-6736-4dfc-bfda-035041671b2b service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:46.132625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d97134c7-6736-4dfc-bfda-035041671b2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:46.179227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df5794-3b0b-47c3-a947-49b399816af9 service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:46.194236 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053205b1-d46f-43dc-8da1-4f7d85b2a76e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:46.195556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053205b1-d46f-43dc-8da1-4f7d85b2a76e service placement] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:46.195845 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053205b1-d46f-43dc-8da1-4f7d85b2a76e service placement] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:46.196120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053205b1-d46f-43dc-8da1-4f7d85b2a76e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:46.198039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-053205b1-d46f-43dc-8da1-4f7d85b2a76e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:46.213990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053205b1-d46f-43dc-8da1-4f7d85b2a76e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:46.215746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053205b1-d46f-43dc-8da1-4f7d85b2a76e service placement] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:46.228941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1261/2525] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:46.251776 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97134c7-6736-4dfc-bfda-035041671b2b service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:46.261234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce4eab-9d66-4fae-84b4-c6e3f3d5988c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:46.262189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce4eab-9d66-4fae-84b4-c6e3f3d5988c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:46.262610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce4eab-9d66-4fae-84b4-c6e3f3d5988c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:46.263028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce4eab-9d66-4fae-84b4-c6e3f3d5988c admin admin] PATH_INFO: `/v3/users/1530c2946ff3486daf7c8b2b70c1a3b6` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:46.274612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce4eab-9d66-4fae-84b4-c6e3f3d5988c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1530c2946ff3486daf7c8b2b70c1a3b6)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:46.276220 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce4eab-9d66-4fae-84b4-c6e3f3d5988c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:46.345118 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-efce4eab-9d66-4fae-84b4-c6e3f3d5988c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1530c2946ff3486daf7c8b2b70c1a3b6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:46.356973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-efce4eab-9d66-4fae-84b4-c6e3f3d5988c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1530c2946ff3486daf7c8b2b70c1a3b6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:46.358327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-efce4eab-9d66-4fae-84b4-c6e3f3d5988c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1530c2946ff3486daf7c8b2b70c1a3b6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:46.375952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-efce4eab-9d66-4fae-84b4-c6e3f3d5988c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1530c2946ff3486daf7c8b2b70c1a3b6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:46.376724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-efce4eab-9d66-4fae-84b4-c6e3f3d5988c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1530c2946ff3486daf7c8b2b70c1a3b6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:46.397994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1265/2526] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:46 2022] DELETE /identity/v3/users/1530c2946ff3486daf7c8b2b70c1a3b6 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:46.623643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053205b1-d46f-43dc-8da1-4f7d85b2a76e service placement] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:46.642169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053205b1-d46f-43dc-8da1-4f7d85b2a76e service placement] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:46.718963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2356d6-973b-4f5f-a92d-915fa19568e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:46.720158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2356d6-973b-4f5f-a92d-915fa19568e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:46.720607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2356d6-973b-4f5f-a92d-915fa19568e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:46.721071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2356d6-973b-4f5f-a92d-915fa19568e4 admin admin] PATH_INFO: `/v3/projects/7713ff3ccc8546b68a89f2e2087d5abf` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:46.722677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2356d6-973b-4f5f-a92d-915fa19568e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7713ff3ccc8546b68a89f2e2087d5abf)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:46.725091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2356d6-973b-4f5f-a92d-915fa19568e4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:46.742451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:46.799794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7e2356d6-973b-4f5f-a92d-915fa19568e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7713ff3ccc8546b68a89f2e2087d5abf', 'request_id': 'req-7e2356d6-973b-4f5f-a92d-915fa19568e4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:46.811229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7e2356d6-973b-4f5f-a92d-915fa19568e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7713ff3ccc8546b68a89f2e2087d5abf', 'request_id': 'req-7e2356d6-973b-4f5f-a92d-915fa19568e4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:46.819268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7e2356d6-973b-4f5f-a92d-915fa19568e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7713ff3ccc8546b68a89f2e2087d5abf', 'request_id': 'req-7e2356d6-973b-4f5f-a92d-915fa19568e4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:46.833413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7e2356d6-973b-4f5f-a92d-915fa19568e4 admin admin] The token cache is being invalidate because project 7713ff3ccc8546b68a89f2e2087d5abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:46.834376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7e2356d6-973b-4f5f-a92d-915fa19568e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:46.834689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7e2356d6-973b-4f5f-a92d-915fa19568e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:46.836672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1262/2527] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:46 2022] DELETE /identity/v3/projects/7713ff3ccc8546b68a89f2e2087d5abf => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:46.843087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa9c37-2e81-410f-86ee-566d05928ae2 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:46.843087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa9c37-2e81-410f-86ee-566d05928ae2 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:46.843087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa9c37-2e81-410f-86ee-566d05928ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:46.867048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-fcaa9c37-2e81-410f-86ee-566d05928ae2 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:46.868434 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fcaa9c37-2e81-410f-86ee-566d05928ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:46.947670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1266/2528] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:46.956285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58de7f-d13c-4806-a102-bdf5a2ac367f None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:46.956698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58de7f-d13c-4806-a102-bdf5a2ac367f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:46.957213 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58de7f-d13c-4806-a102-bdf5a2ac367f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:46.987495 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-7e58de7f-d13c-4806-a102-bdf5a2ac367f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:46.988882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7e58de7f-d13c-4806-a102-bdf5a2ac367f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:46.997515 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa9c37-2e81-410f-86ee-566d05928ae2 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:46.999653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fcaa9c37-2e81-410f-86ee-566d05928ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:47.009928 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:47.015048 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1263/2529] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:47.017080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58de7f-d13c-4806-a102-bdf5a2ac367f None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:47.019010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa9c37-2e81-410f-86ee-566d05928ae2 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:47.026543 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:47.027503 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.028028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.028371 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] PATH_INFO: `/v3/users/980ab18eb5c646378d738079621a0dbc` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.029252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58de7f-d13c-4806-a102-bdf5a2ac367f None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:47.030346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=980ab18eb5c646378d738079621a0dbc)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:47.033780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:47.034829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2425b-ca7e-4979-9f81-076fc3fa6d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:47.035441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2425b-ca7e-4979-9f81-076fc3fa6d9f admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.035793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2425b-ca7e-4979-9f81-076fc3fa6d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.036209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2425b-ca7e-4979-9f81-076fc3fa6d9f admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.037941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2425b-ca7e-4979-9f81-076fc3fa6d9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:47.037941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2425b-ca7e-4979-9f81-076fc3fa6d9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:47.040030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2425b-ca7e-4979-9f81-076fc3fa6d9f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:47.055405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1264/2530] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:47.061265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2425b-ca7e-4979-9f81-076fc3fa6d9f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:47.086817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2425b-ca7e-4979-9f81-076fc3fa6d9f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:47.093411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8f0e2-bd91-484e-99ed-048e249a8ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:47.094374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8f0e2-bd91-484e-99ed-048e249a8ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.094800 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8f0e2-bd91-484e-99ed-048e249a8ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.095667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8f0e2-bd91-484e-99ed-048e249a8ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.096715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8f0e2-bd91-484e-99ed-048e249a8ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:47.099242 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8f0e2-bd91-484e-99ed-048e249a8ec9 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:47.127427 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '980ab18eb5c646378d738079621a0dbc'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:47.139855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1265/2531] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:47 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:47.141327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8f0e2-bd91-484e-99ed-048e249a8ec9 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:47.151724 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '980ab18eb5c646378d738079621a0dbc'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:47.152870 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '980ab18eb5c646378d738079621a0dbc'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:47.174590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '980ab18eb5c646378d738079621a0dbc'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:47.175455 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '980ab18eb5c646378d738079621a0dbc'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:47.193846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1267/2532] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:46 2022] DELETE /identity/v3/users/980ab18eb5c646378d738079621a0dbc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:55:47.281701 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8f0e2-bd91-484e-99ed-048e249a8ec9 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:47.420848 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e7043-4dfe-4431-9ffc-92ac8a9cb1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:47.421583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e7043-4dfe-4431-9ffc-92ac8a9cb1f1 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.421963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e7043-4dfe-4431-9ffc-92ac8a9cb1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.422334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e7043-4dfe-4431-9ffc-92ac8a9cb1f1 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.423206 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e7043-4dfe-4431-9ffc-92ac8a9cb1f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:47.425518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e7043-4dfe-4431-9ffc-92ac8a9cb1f1 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:47.433400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:47.435653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:47.479192 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:47.479627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d027212-c134-4d50-81f3-01f2b7f41659 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:47.497009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:47.497662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.497941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.498220 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8 admin admin] PATH_INFO: `/v3/projects/c4914faafb494e5bb323114853f25c54` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.499520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4914faafb494e5bb323114853f25c54)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:47.499713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1266/2533] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:47 2022] POST /identity/v3/users => generated 287 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:55:47.503727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:47.504518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e7043-4dfe-4431-9ffc-92ac8a9cb1f1 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:47.509007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-311e7043-4dfe-4431-9ffc-92ac8a9cb1f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:47.525369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e7043-4dfe-4431-9ffc-92ac8a9cb1f1 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:47.532898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e05f8b-9e85-4bb5-810b-864162cc08c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:47.534624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e05f8b-9e85-4bb5-810b-864162cc08c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.535261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e05f8b-9e85-4bb5-810b-864162cc08c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.535700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e05f8b-9e85-4bb5-810b-864162cc08c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.536457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e05f8b-9e85-4bb5-810b-864162cc08c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:47.539891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e05f8b-9e85-4bb5-810b-864162cc08c3 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:47.559544 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1267/2534] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:47.561315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e05f8b-9e85-4bb5-810b-864162cc08c3 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:47.584829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e05f8b-9e85-4bb5-810b-864162cc08c3 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:47.601150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4914faafb494e5bb323114853f25c54', 'request_id': 'req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:47.614193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4914faafb494e5bb323114853f25c54', 'request_id': 'req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:47.623594 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4914faafb494e5bb323114853f25c54', 'request_id': 'req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:47.643245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8 admin admin] The token cache is being invalidate because project c4914faafb494e5bb323114853f25c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:47.643739 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:47.649301 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c80d617c-af3a-48cd-b14f-7dfc6699c0e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:47.650622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1268/2535] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:47 2022] DELETE /identity/v3/projects/c4914faafb494e5bb323114853f25c54 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:47.655286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8050566-073e-4d1e-b693-fb71651a80f7 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.655642 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8050566-073e-4d1e-b693-fb71651a80f7 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.656007 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8050566-073e-4d1e-b693-fb71651a80f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.682621 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f8050566-073e-4d1e-b693-fb71651a80f7 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:47.684032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f8050566-073e-4d1e-b693-fb71651a80f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:47.685190 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f2091-d257-4051-813a-af1ab2d80719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:47.685856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f2091-d257-4051-813a-af1ab2d80719 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.686202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f2091-d257-4051-813a-af1ab2d80719 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.686541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f2091-d257-4051-813a-af1ab2d80719 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.687297 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f2091-d257-4051-813a-af1ab2d80719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:47.689055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f2091-d257-4051-813a-af1ab2d80719 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:47.692695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1269/2536] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:47.700079 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1268/2537] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:47.701302 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d13e5c-6cf0-42ff-b201-71d09027f453 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.701592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d13e5c-6cf0-42ff-b201-71d09027f453 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.701924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d13e5c-6cf0-42ff-b201-71d09027f453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.708218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f2091-d257-4051-813a-af1ab2d80719 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:47.710280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a47f2091-d257-4051-813a-af1ab2d80719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:47.722856 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a6d13e5c-6cf0-42ff-b201-71d09027f453 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:47.724159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a6d13e5c-6cf0-42ff-b201-71d09027f453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:47.727007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f2091-d257-4051-813a-af1ab2d80719 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:47.731800 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a81cf-0ed8-4304-8448-ec43e9f7a758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:47.732537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a81cf-0ed8-4304-8448-ec43e9f7a758 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.732884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a81cf-0ed8-4304-8448-ec43e9f7a758 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.733268 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a81cf-0ed8-4304-8448-ec43e9f7a758 admin admin] PATH_INFO: `/v3/projects/0fa14e21b82942ee8ff3fdb25ccf1ed0/users/71a5a25b4a704fe0b321959e4f812310/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.735841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1270/2538] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:47.739992 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d13e5c-6cf0-42ff-b201-71d09027f453 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:47.750363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d13e5c-6cf0-42ff-b201-71d09027f453 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:47.752841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a81cf-0ed8-4304-8448-ec43e9f7a758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fa14e21b82942ee8ff3fdb25ccf1ed0, user_id=71a5a25b4a704fe0b321959e4f812310, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:47.761044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5566d2-21b9-42a5-9d5f-3db0f9095d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:47.761044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5566d2-21b9-42a5-9d5f-3db0f9095d71 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.761280 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a81cf-0ed8-4304-8448-ec43e9f7a758 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:47.761391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5566d2-21b9-42a5-9d5f-3db0f9095d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.761391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5566d2-21b9-42a5-9d5f-3db0f9095d71 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.761391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5566d2-21b9-42a5-9d5f-3db0f9095d71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:47.761391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5566d2-21b9-42a5-9d5f-3db0f9095d71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:47.761391 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5566d2-21b9-42a5-9d5f-3db0f9095d71 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:47.774734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1269/2539] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:47 2022] PUT /identity/v3/projects/0fa14e21b82942ee8ff3fdb25ccf1ed0/users/71a5a25b4a704fe0b321959e4f812310/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:47.778305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1271/2540] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:55:47.779704 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a81cf-0ed8-4304-8448-ec43e9f7a758 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:47.843473 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba2efb-a942-4b67-ab44-9064b58e5145 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.844548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba2efb-a942-4b67-ab44-9064b58e5145 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.845789 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba2efb-a942-4b67-ab44-9064b58e5145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.859448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a81cf-0ed8-4304-8448-ec43e9f7a758 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:47.864534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3831e74-2b37-49c1-987a-62d921e6dd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:47.865114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3831e74-2b37-49c1-987a-62d921e6dd32 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.865421 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3831e74-2b37-49c1-987a-62d921e6dd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.866481 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3831e74-2b37-49c1-987a-62d921e6dd32 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.867264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3831e74-2b37-49c1-987a-62d921e6dd32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:47.869112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3831e74-2b37-49c1-987a-62d921e6dd32 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:47.879766 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-9bba2efb-a942-4b67-ab44-9064b58e5145 None None] MFA Rules not processed for user `71a5a25b4a704fe0b321959e4f812310`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:47.882415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9bba2efb-a942-4b67-ab44-9064b58e5145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:47.903376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1270/2541] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:47 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:55:47.907662 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3831e74-2b37-49c1-987a-62d921e6dd32 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:47.909155 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:47.969892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1272/2542] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:47.976697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:47.988009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba2efb-a942-4b67-ab44-9064b58e5145 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:47.992957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9bba2efb-a942-4b67-ab44-9064b58e5145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:48.001941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3831e74-2b37-49c1-987a-62d921e6dd32 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.012795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609a537-3c89-476c-8413-2f20453dcd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.014306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609a537-3c89-476c-8413-2f20453dcd5e admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.014639 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609a537-3c89-476c-8413-2f20453dcd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.015038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609a537-3c89-476c-8413-2f20453dcd5e admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.015727 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609a537-3c89-476c-8413-2f20453dcd5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.018030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609a537-3c89-476c-8413-2f20453dcd5e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.051375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1271/2543] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:47 2022] POST /identity/v3/users => generated 285 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:55:48.054185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609a537-3c89-476c-8413-2f20453dcd5e admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:48.066870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609a537-3c89-476c-8413-2f20453dcd5e admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.078229 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba2efb-a942-4b67-ab44-9064b58e5145 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.080967 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94011663-3777-4900-b79c-5c42a0dec259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.081600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94011663-3777-4900-b79c-5c42a0dec259 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.081884 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94011663-3777-4900-b79c-5c42a0dec259 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.082287 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94011663-3777-4900-b79c-5c42a0dec259 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.082950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94011663-3777-4900-b79c-5c42a0dec259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.085054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94011663-3777-4900-b79c-5c42a0dec259 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.098743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1272/2544] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:55:48.103801 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94011663-3777-4900-b79c-5c42a0dec259 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:48.114783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94011663-3777-4900-b79c-5c42a0dec259 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.122224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b12ca-56d1-4e45-9986-9a80cf97fc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.122863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b12ca-56d1-4e45-9986-9a80cf97fc65 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.123288 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b12ca-56d1-4e45-9986-9a80cf97fc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.123435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b12ca-56d1-4e45-9986-9a80cf97fc65 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.124083 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b12ca-56d1-4e45-9986-9a80cf97fc65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.126815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b12ca-56d1-4e45-9986-9a80cf97fc65 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.142598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1273/2545] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:55:48.143398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b12ca-56d1-4e45-9986-9a80cf97fc65 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:48.146245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe424e0-3b31-4f13-9909-0cdd9bc99965 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.147639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe424e0-3b31-4f13-9909-0cdd9bc99965 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.148357 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe424e0-3b31-4f13-9909-0cdd9bc99965 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.148357 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe424e0-3b31-4f13-9909-0cdd9bc99965 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.156519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b12ca-56d1-4e45-9986-9a80cf97fc65 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.163727 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1fc63-f899-4fa4-ac90-e23633199a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.164398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1fc63-f899-4fa4-ac90-e23633199a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.164676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1fc63-f899-4fa4-ac90-e23633199a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.165015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1fc63-f899-4fa4-ac90-e23633199a4b admin admin] PATH_INFO: `/v3/projects/9d33a1d09fa54fc7b852597772c78751/users/deaad1ec377f4786945c82172a83f1ea/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.196339 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1fc63-f899-4fa4-ac90-e23633199a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d33a1d09fa54fc7b852597772c78751, user_id=deaad1ec377f4786945c82172a83f1ea, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.201859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1fc63-f899-4fa4-ac90-e23633199a4b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.219924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1274/2546] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:48 2022] PUT /identity/v3/projects/9d33a1d09fa54fc7b852597772c78751/users/deaad1ec377f4786945c82172a83f1ea/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:48.229085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe424e0-3b31-4f13-9909-0cdd9bc99965 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.231158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe424e0-3b31-4f13-9909-0cdd9bc99965 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.231445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83265cf-4b29-406b-911a-8326e448ebb9 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.231856 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83265cf-4b29-406b-911a-8326e448ebb9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.232136 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83265cf-4b29-406b-911a-8326e448ebb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.244951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1273/2547] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:48.269295 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-a83265cf-4b29-406b-911a-8326e448ebb9 None None] MFA Rules not processed for user `deaad1ec377f4786945c82172a83f1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:48.270760 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a83265cf-4b29-406b-911a-8326e448ebb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:48.274472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe424e0-3b31-4f13-9909-0cdd9bc99965 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:48.288252 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe424e0-3b31-4f13-9909-0cdd9bc99965 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.326172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:48.359417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1275/2548] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:48.363709 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b22a3-60e6-4238-a38f-45f2abdaadb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.364240 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83265cf-4b29-406b-911a-8326e448ebb9 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:48.364490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b22a3-60e6-4238-a38f-45f2abdaadb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.364768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b22a3-60e6-4238-a38f-45f2abdaadb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.365036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b22a3-60e6-4238-a38f-45f2abdaadb5 admin admin] PATH_INFO: `/v3/users/71a5a25b4a704fe0b321959e4f812310` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.366240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b22a3-60e6-4238-a38f-45f2abdaadb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71a5a25b4a704fe0b321959e4f812310)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.368880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b22a3-60e6-4238-a38f-45f2abdaadb5 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.378013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83265cf-4b29-406b-911a-8326e448ebb9 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.385031 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bf614-cf5e-4ad1-8727-533ed6893b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.385941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bf614-cf5e-4ad1-8727-533ed6893b12 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.386084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bf614-cf5e-4ad1-8727-533ed6893b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.386413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bf614-cf5e-4ad1-8727-533ed6893b12 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.387224 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bf614-cf5e-4ad1-8727-533ed6893b12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.389391 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bf614-cf5e-4ad1-8727-533ed6893b12 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.426542 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1276/2549] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:48 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:48.430742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bf614-cf5e-4ad1-8727-533ed6893b12 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:48.433048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d40b22a3-60e6-4238-a38f-45f2abdaadb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71a5a25b4a704fe0b321959e4f812310'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:48.441689 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d40b22a3-60e6-4238-a38f-45f2abdaadb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71a5a25b4a704fe0b321959e4f812310'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:48.442390 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d40b22a3-60e6-4238-a38f-45f2abdaadb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71a5a25b4a704fe0b321959e4f812310'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:48.457799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d40b22a3-60e6-4238-a38f-45f2abdaadb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71a5a25b4a704fe0b321959e4f812310'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:48.458647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d40b22a3-60e6-4238-a38f-45f2abdaadb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71a5a25b4a704fe0b321959e4f812310'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:48.474534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1274/2550] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:48 2022] DELETE /identity/v3/users/71a5a25b4a704fe0b321959e4f812310 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:48.484526 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406324e3-db44-48cb-a0e9-885f0e07bbeb None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.484909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406324e3-db44-48cb-a0e9-885f0e07bbeb None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.485191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406324e3-db44-48cb-a0e9-885f0e07bbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.495838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:48.509866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-406324e3-db44-48cb-a0e9-885f0e07bbeb None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:48.510229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bf614-cf5e-4ad1-8727-533ed6893b12 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.511111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-406324e3-db44-48cb-a0e9-885f0e07bbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:48.591706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d1259-d647-40af-86ce-5de2baf9d6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.592337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d1259-d647-40af-86ce-5de2baf9d6a6 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.592699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d1259-d647-40af-86ce-5de2baf9d6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.592975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d1259-d647-40af-86ce-5de2baf9d6a6 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.593762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d1259-d647-40af-86ce-5de2baf9d6a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.595812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d1259-d647-40af-86ce-5de2baf9d6a6 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.619097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1275/2551] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:48.626665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1277/2552] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:48 2022] POST /identity/v3/users => generated 291 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:55:48.632074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d1259-d647-40af-86ce-5de2baf9d6a6 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:48.632762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406324e3-db44-48cb-a0e9-885f0e07bbeb None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:48.635179 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-120d1259-d647-40af-86ce-5de2baf9d6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:48.635619 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-406324e3-db44-48cb-a0e9-885f0e07bbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:48.659309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406324e3-db44-48cb-a0e9-885f0e07bbeb None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.669653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559f21e-4f70-4836-b8c5-d459bd1624b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.671087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559f21e-4f70-4836-b8c5-d459bd1624b1 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.671773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559f21e-4f70-4836-b8c5-d459bd1624b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.672251 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559f21e-4f70-4836-b8c5-d459bd1624b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.673865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559f21e-4f70-4836-b8c5-d459bd1624b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.678692 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559f21e-4f70-4836-b8c5-d459bd1624b1 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.703198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1276/2553] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:48.708827 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559f21e-4f70-4836-b8c5-d459bd1624b1 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:48.724727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559f21e-4f70-4836-b8c5-d459bd1624b1 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.729843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d1259-d647-40af-86ce-5de2baf9d6a6 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.730910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79da78-b790-40fa-a91c-5af4deab9af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.731686 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79da78-b790-40fa-a91c-5af4deab9af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.732153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79da78-b790-40fa-a91c-5af4deab9af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.732627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79da78-b790-40fa-a91c-5af4deab9af4 admin admin] PATH_INFO: `/v3/projects/b5ec60c5a3e34ff78c94b7ef06adcc12/users/35a32b264db74b33ae621b886bdef7e8/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.734511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4123e-d187-43f5-a39d-17c2f8f19e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.735319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4123e-d187-43f5-a39d-17c2f8f19e7f service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.736108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4123e-d187-43f5-a39d-17c2f8f19e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.736276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4123e-d187-43f5-a39d-17c2f8f19e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.753994 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4123e-d187-43f5-a39d-17c2f8f19e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.756812 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4123e-d187-43f5-a39d-17c2f8f19e7f service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.761734 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79da78-b790-40fa-a91c-5af4deab9af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ec60c5a3e34ff78c94b7ef06adcc12, user_id=35a32b264db74b33ae621b886bdef7e8, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.768725 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79da78-b790-40fa-a91c-5af4deab9af4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.780835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1278/2554] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:48.794572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1277/2555] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:48 2022] PUT /identity/v3/projects/b5ec60c5a3e34ff78c94b7ef06adcc12/users/35a32b264db74b33ae621b886bdef7e8/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:48.799309 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4123e-d187-43f5-a39d-17c2f8f19e7f service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:48.810632 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4123e-d187-43f5-a39d-17c2f8f19e7f service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.895547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33e283-ded0-47f6-98e4-eddcdd830298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.896162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33e283-ded0-47f6-98e4-eddcdd830298 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.897017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33e283-ded0-47f6-98e4-eddcdd830298 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.897888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33e283-ded0-47f6-98e4-eddcdd830298 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.898732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33e283-ded0-47f6-98e4-eddcdd830298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.900670 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33e283-ded0-47f6-98e4-eddcdd830298 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.920640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1279/2556] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:55:48.924494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79da78-b790-40fa-a91c-5af4deab9af4 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:48.947262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79da78-b790-40fa-a91c-5af4deab9af4 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:48.956602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ed978-e6b3-48c6-9aca-c2ef5707d07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:48.958086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ed978-e6b3-48c6-9aca-c2ef5707d07b admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:48.958394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ed978-e6b3-48c6-9aca-c2ef5707d07b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:48.959553 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ed978-e6b3-48c6-9aca-c2ef5707d07b admin admin] PATH_INFO: `/v3/projects/b5ec60c5a3e34ff78c94b7ef06adcc12/users/35a32b264db74b33ae621b886bdef7e8/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:48.965190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ed978-e6b3-48c6-9aca-c2ef5707d07b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ec60c5a3e34ff78c94b7ef06adcc12, user_id=35a32b264db74b33ae621b886bdef7e8, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:48.970216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ed978-e6b3-48c6-9aca-c2ef5707d07b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:48.990578 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1278/2557] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:48 2022] PUT /identity/v3/projects/b5ec60c5a3e34ff78c94b7ef06adcc12/users/35a32b264db74b33ae621b886bdef7e8/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:49.002290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535731a-9262-4a31-9ba3-5ca96feacb57 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:49.002702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535731a-9262-4a31-9ba3-5ca96feacb57 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:49.003095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535731a-9262-4a31-9ba3-5ca96feacb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:49.044889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-3535731a-9262-4a31-9ba3-5ca96feacb57 None None] MFA Rules not processed for user `35a32b264db74b33ae621b886bdef7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:49.046381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3535731a-9262-4a31-9ba3-5ca96feacb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:49.065379 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:49.133681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1280/2558] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:49.143299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ed978-e6b3-48c6-9aca-c2ef5707d07b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:49.145221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7b6ed978-e6b3-48c6-9aca-c2ef5707d07b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:49.151171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:49.166954 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535731a-9262-4a31-9ba3-5ca96feacb57 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:49.168926 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-3535731a-9262-4a31-9ba3-5ca96feacb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:49.238881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ed978-e6b3-48c6-9aca-c2ef5707d07b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:49.244912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade62e0-1af1-4ec9-ad75-049c909f6339 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:49.246064 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade62e0-1af1-4ec9-ad75-049c909f6339 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:49.246569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade62e0-1af1-4ec9-ad75-049c909f6339 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:49.246996 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade62e0-1af1-4ec9-ad75-049c909f6339 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:49.258467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade62e0-1af1-4ec9-ad75-049c909f6339 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:49.260296 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade62e0-1af1-4ec9-ad75-049c909f6339 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:49.273565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1279/2559] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:49.304134 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535731a-9262-4a31-9ba3-5ca96feacb57 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:49.312913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfdfc8-191f-42c3-93f0-09c69036bd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:49.314123 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfdfc8-191f-42c3-93f0-09c69036bd1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:49.314570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfdfc8-191f-42c3-93f0-09c69036bd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:49.315485 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfdfc8-191f-42c3-93f0-09c69036bd1f admin admin] PATH_INFO: `/v3/projects/0fa14e21b82942ee8ff3fdb25ccf1ed0` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:49.317007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfdfc8-191f-42c3-93f0-09c69036bd1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fa14e21b82942ee8ff3fdb25ccf1ed0)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:49.321629 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfdfc8-191f-42c3-93f0-09c69036bd1f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:49.406909 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-42bfdfc8-191f-42c3-93f0-09c69036bd1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fa14e21b82942ee8ff3fdb25ccf1ed0', 'request_id': 'req-42bfdfc8-191f-42c3-93f0-09c69036bd1f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:49.419674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-42bfdfc8-191f-42c3-93f0-09c69036bd1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fa14e21b82942ee8ff3fdb25ccf1ed0', 'request_id': 'req-42bfdfc8-191f-42c3-93f0-09c69036bd1f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:49.431727 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-42bfdfc8-191f-42c3-93f0-09c69036bd1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fa14e21b82942ee8ff3fdb25ccf1ed0', 'request_id': 'req-42bfdfc8-191f-42c3-93f0-09c69036bd1f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:49.448725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-42bfdfc8-191f-42c3-93f0-09c69036bd1f admin admin] The token cache is being invalidate because project 0fa14e21b82942ee8ff3fdb25ccf1ed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:49.449241 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-42bfdfc8-191f-42c3-93f0-09c69036bd1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:49.450313 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-42bfdfc8-191f-42c3-93f0-09c69036bd1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:49.452677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1281/2560] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:49 2022] DELETE /identity/v3/projects/0fa14e21b82942ee8ff3fdb25ccf1ed0 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:49.459837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f8d58-90c6-4040-8c42-c7e46465c648 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:49.460131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f8d58-90c6-4040-8c42-c7e46465c648 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:49.460508 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f8d58-90c6-4040-8c42-c7e46465c648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:49.486308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-194f8d58-90c6-4040-8c42-c7e46465c648 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:49.487591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-194f8d58-90c6-4040-8c42-c7e46465c648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:49.595181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1280/2561] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:49.608310 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01de1e0-ae90-4604-be3f-b5c73969015b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:49.608851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01de1e0-ae90-4604-be3f-b5c73969015b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:49.609230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01de1e0-ae90-4604-be3f-b5c73969015b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:49.645837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e01de1e0-ae90-4604-be3f-b5c73969015b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:49.648389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e01de1e0-ae90-4604-be3f-b5c73969015b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:49.661604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f8d58-90c6-4040-8c42-c7e46465c648 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:49.667976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1282/2562] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:49.668210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-194f8d58-90c6-4040-8c42-c7e46465c648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:49.669577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01de1e0-ae90-4604-be3f-b5c73969015b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:49.681508 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f8d58-90c6-4040-8c42-c7e46465c648 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:49.685155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01de1e0-ae90-4604-be3f-b5c73969015b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:49.686982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5ba64-d3e1-40bc-9995-1da6ebe6fa37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:49.687900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5ba64-d3e1-40bc-9995-1da6ebe6fa37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:49.688125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5ba64-d3e1-40bc-9995-1da6ebe6fa37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:49.688464 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5ba64-d3e1-40bc-9995-1da6ebe6fa37 admin admin] PATH_INFO: `/v3/users/5ef138e7ba8441e4ac98e193737cfa75` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:49.689898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5ba64-d3e1-40bc-9995-1da6ebe6fa37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ef138e7ba8441e4ac98e193737cfa75)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:49.692150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5ba64-d3e1-40bc-9995-1da6ebe6fa37 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:49.692355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8ac9d-8fc8-495f-b4fc-b314a3cb0086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:49.693388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8ac9d-8fc8-495f-b4fc-b314a3cb0086 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:49.693892 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8ac9d-8fc8-495f-b4fc-b314a3cb0086 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:49.694581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8ac9d-8fc8-495f-b4fc-b314a3cb0086 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:49.699238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8ac9d-8fc8-495f-b4fc-b314a3cb0086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:49.699768 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8ac9d-8fc8-495f-b4fc-b314a3cb0086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:49.702013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8ac9d-8fc8-495f-b4fc-b314a3cb0086 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:49.717578 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1283/2563] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:49.723878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8ac9d-8fc8-495f-b4fc-b314a3cb0086 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:49.737271 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8ac9d-8fc8-495f-b4fc-b314a3cb0086 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:49.742583 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31a789-fa70-45a6-9a90-83d6ba7a0742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:49.743222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31a789-fa70-45a6-9a90-83d6ba7a0742 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:49.743496 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31a789-fa70-45a6-9a90-83d6ba7a0742 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:49.743767 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31a789-fa70-45a6-9a90-83d6ba7a0742 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:49.744415 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31a789-fa70-45a6-9a90-83d6ba7a0742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:49.746796 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31a789-fa70-45a6-9a90-83d6ba7a0742 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:49.762890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6ad5ba64-d3e1-40bc-9995-1da6ebe6fa37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ef138e7ba8441e4ac98e193737cfa75'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:49.773648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6ad5ba64-d3e1-40bc-9995-1da6ebe6fa37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ef138e7ba8441e4ac98e193737cfa75'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:49.775746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6ad5ba64-d3e1-40bc-9995-1da6ebe6fa37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ef138e7ba8441e4ac98e193737cfa75'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:49.786986 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1284/2564] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:49 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:55:49.796101 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31a789-fa70-45a6-9a90-83d6ba7a0742 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:49.798267 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6ad5ba64-d3e1-40bc-9995-1da6ebe6fa37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ef138e7ba8441e4ac98e193737cfa75'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:49.799065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-6ad5ba64-d3e1-40bc-9995-1da6ebe6fa37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ef138e7ba8441e4ac98e193737cfa75'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:49.800508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2a31a789-fa70-45a6-9a90-83d6ba7a0742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:49.809708 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:49.830065 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1281/2565] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:49 2022] DELETE /identity/v3/users/5ef138e7ba8441e4ac98e193737cfa75 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:49.833298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ba4e1-13f9-40cd-a148-af348e954a30 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:49.833855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ba4e1-13f9-40cd-a148-af348e954a30 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:49.834522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ba4e1-13f9-40cd-a148-af348e954a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:49.881728 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a61ba4e1-13f9-40cd-a148-af348e954a30 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:49.883527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a61ba4e1-13f9-40cd-a148-af348e954a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:49.909609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:49.932791 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31a789-fa70-45a6-9a90-83d6ba7a0742 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:50.013951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1282/2566] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:50.028526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd0864-891c-4dc9-a7d3-d8e7b3ba8ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:50.029056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ba4e1-13f9-40cd-a148-af348e954a30 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:50.029466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd0864-891c-4dc9-a7d3-d8e7b3ba8ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:50.029938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd0864-891c-4dc9-a7d3-d8e7b3ba8ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:50.030529 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd0864-891c-4dc9-a7d3-d8e7b3ba8ed2 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:50.031299 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd0864-891c-4dc9-a7d3-d8e7b3ba8ed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:50.031491 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a61ba4e1-13f9-40cd-a148-af348e954a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:50.033895 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd0864-891c-4dc9-a7d3-d8e7b3ba8ed2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:50.067195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1285/2567] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:49 2022] POST /identity/v3/users => generated 291 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:50.072992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd0864-891c-4dc9-a7d3-d8e7b3ba8ed2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:50.104219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd0864-891c-4dc9-a7d3-d8e7b3ba8ed2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:50.112690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46db90-4fa5-4ed1-a123-0603dd361090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:50.113654 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46db90-4fa5-4ed1-a123-0603dd361090 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:50.116136 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46db90-4fa5-4ed1-a123-0603dd361090 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:50.116136 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46db90-4fa5-4ed1-a123-0603dd361090 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:50.116769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46db90-4fa5-4ed1-a123-0603dd361090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:50.120199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46db90-4fa5-4ed1-a123-0603dd361090 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:50.145141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1286/2568] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:50.150724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46db90-4fa5-4ed1-a123-0603dd361090 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:50.154237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ba4e1-13f9-40cd-a148-af348e954a30 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:50.163902 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46db90-4fa5-4ed1-a123-0603dd361090 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:50.164236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c271a-8154-4a2f-b388-e974350eddab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:50.166046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c271a-8154-4a2f-b388-e974350eddab service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:50.166755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c271a-8154-4a2f-b388-e974350eddab service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:50.167799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c271a-8154-4a2f-b388-e974350eddab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:50.169657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ff6f1-4682-43d0-9bb8-309b406ea16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:50.170680 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ff6f1-4682-43d0-9bb8-309b406ea16d admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:50.170968 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ff6f1-4682-43d0-9bb8-309b406ea16d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:50.171340 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ff6f1-4682-43d0-9bb8-309b406ea16d admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:50.172159 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ff6f1-4682-43d0-9bb8-309b406ea16d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:50.174488 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ff6f1-4682-43d0-9bb8-309b406ea16d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:50.187916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c271a-8154-4a2f-b388-e974350eddab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:50.190787 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c271a-8154-4a2f-b388-e974350eddab service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:50.198871 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1287/2569] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:50.200239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ff6f1-4682-43d0-9bb8-309b406ea16d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:50.212377 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1283/2570] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:50.214248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ff6f1-4682-43d0-9bb8-309b406ea16d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:50.220537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a45c3-be2a-4dea-b2d4-34c3019f7492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:50.221286 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a45c3-be2a-4dea-b2d4-34c3019f7492 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:50.221604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a45c3-be2a-4dea-b2d4-34c3019f7492 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:50.221876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a45c3-be2a-4dea-b2d4-34c3019f7492 admin admin] PATH_INFO: `/v3/projects/fbdd47c941404548bf5371c35e518346/users/488e41d766134f3aa7c956816d01df06/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:50.249909 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a45c3-be2a-4dea-b2d4-34c3019f7492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbdd47c941404548bf5371c35e518346, user_id=488e41d766134f3aa7c956816d01df06, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:50.254088 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a45c3-be2a-4dea-b2d4-34c3019f7492 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:50.274647 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1288/2571] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:50 2022] PUT /identity/v3/projects/fbdd47c941404548bf5371c35e518346/users/488e41d766134f3aa7c956816d01df06/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:50.286561 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d267e2-7508-488e-b1c6-158c022f2df5 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:50.286561 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d267e2-7508-488e-b1c6-158c022f2df5 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:50.288183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d267e2-7508-488e-b1c6-158c022f2df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:50.312727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-87d267e2-7508-488e-b1c6-158c022f2df5 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:50.314096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-87d267e2-7508-488e-b1c6-158c022f2df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:50.426841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1284/2572] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:50.438578 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a45c3-be2a-4dea-b2d4-34c3019f7492 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:50.441747 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-635a45c3-be2a-4dea-b2d4-34c3019f7492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:50.547023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a45c3-be2a-4dea-b2d4-34c3019f7492 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:50.561256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3ca7b-3568-4f5b-94d4-ef205da2aaeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:50.561887 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3ca7b-3568-4f5b-94d4-ef205da2aaeb service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:50.562165 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3ca7b-3568-4f5b-94d4-ef205da2aaeb service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:50.562672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3ca7b-3568-4f5b-94d4-ef205da2aaeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:50.580307 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:50.582183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3ca7b-3568-4f5b-94d4-ef205da2aaeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:50.589912 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3ca7b-3568-4f5b-94d4-ef205da2aaeb service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:50.611595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1289/2573] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:50.635716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d267e2-7508-488e-b1c6-158c022f2df5 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:50.638968 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-87d267e2-7508-488e-b1c6-158c022f2df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:50.654937 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d267e2-7508-488e-b1c6-158c022f2df5 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:50.665842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2355e56-e776-47c7-9067-a75f72a9b8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:50.666969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2355e56-e776-47c7-9067-a75f72a9b8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:50.667535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2355e56-e776-47c7-9067-a75f72a9b8be admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:50.668035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2355e56-e776-47c7-9067-a75f72a9b8be admin admin] PATH_INFO: `/v3/projects/829529842e444ba8aef3aa5b75832ba6` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:50.670153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2355e56-e776-47c7-9067-a75f72a9b8be admin admin] RBAC: Authorizing `identity:delete_project(project_id=829529842e444ba8aef3aa5b75832ba6)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:50.674037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2355e56-e776-47c7-9067-a75f72a9b8be admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:50.768634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c2355e56-e776-47c7-9067-a75f72a9b8be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '829529842e444ba8aef3aa5b75832ba6', 'request_id': 'req-c2355e56-e776-47c7-9067-a75f72a9b8be'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:50.785176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c2355e56-e776-47c7-9067-a75f72a9b8be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '829529842e444ba8aef3aa5b75832ba6', 'request_id': 'req-c2355e56-e776-47c7-9067-a75f72a9b8be'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:50.791591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:50.800119 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c2355e56-e776-47c7-9067-a75f72a9b8be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '829529842e444ba8aef3aa5b75832ba6', 'request_id': 'req-c2355e56-e776-47c7-9067-a75f72a9b8be'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:50.825064 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c2355e56-e776-47c7-9067-a75f72a9b8be admin admin] The token cache is being invalidate because project 829529842e444ba8aef3aa5b75832ba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:50.825548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c2355e56-e776-47c7-9067-a75f72a9b8be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:50.826769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c2355e56-e776-47c7-9067-a75f72a9b8be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:50.834502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1285/2574] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:50 2022] DELETE /identity/v3/projects/829529842e444ba8aef3aa5b75832ba6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:50.837349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3ca7b-3568-4f5b-94d4-ef205da2aaeb service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:50.863699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3ca7b-3568-4f5b-94d4-ef205da2aaeb service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:50.994866 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:50.995715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:50.999321 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:51.000022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] PATH_INFO: `/v3/users/1f98db6164ea4edf83aad5cadf7f3290` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:51.002338 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f98db6164ea4edf83aad5cadf7f3290)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:51.004463 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:51.078483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610f44f-5db6-4376-a91c-3b230af382c3 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:51.079343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610f44f-5db6-4376-a91c-3b230af382c3 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:51.080045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610f44f-5db6-4376-a91c-3b230af382c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:51.118075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f98db6164ea4edf83aad5cadf7f3290'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:51.148646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-d610f44f-5db6-4376-a91c-3b230af382c3 None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:51.150862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d610f44f-5db6-4376-a91c-3b230af382c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:51.160364 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f98db6164ea4edf83aad5cadf7f3290'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:51.163143 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f98db6164ea4edf83aad5cadf7f3290'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:51.164339 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f98db6164ea4edf83aad5cadf7f3290'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:51.184533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f98db6164ea4edf83aad5cadf7f3290'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:51.215823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1290/2575] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:50 2022] DELETE /identity/v3/users/1f98db6164ea4edf83aad5cadf7f3290 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:51.325537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1286/2576] 10.222.0.36 () {58 vars in 967 bytes} [Wed Aug 31 09:55:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:51.362926 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:51.364259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:51.540394 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab820357-c28f-4112-a8eb-7eba6646527d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:51.550189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2fd7d-3eb9-40af-bdda-d51399189be2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:51.551079 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2fd7d-3eb9-40af-bdda-d51399189be2 service swift] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:51.551538 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2fd7d-3eb9-40af-bdda-d51399189be2 service swift] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:51.552446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2fd7d-3eb9-40af-bdda-d51399189be2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:51.577897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:51.617697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610f44f-5db6-4376-a91c-3b230af382c3 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:51.623551 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d610f44f-5db6-4376-a91c-3b230af382c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:51.659413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610f44f-5db6-4376-a91c-3b230af382c3 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:51.672005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2fd7d-3eb9-40af-bdda-d51399189be2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:51.678417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2fd7d-3eb9-40af-bdda-d51399189be2 service swift] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:51.698915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1291/2577] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:55:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:55:51.754451 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad819df-b397-465e-b15b-7c873ca9db1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:51.755595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad819df-b397-465e-b15b-7c873ca9db1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:51.756430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad819df-b397-465e-b15b-7c873ca9db1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:51.757067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad819df-b397-465e-b15b-7c873ca9db1a admin admin] PATH_INFO: `/v3/users/deaad1ec377f4786945c82172a83f1ea` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:51.760606 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad819df-b397-465e-b15b-7c873ca9db1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=deaad1ec377f4786945c82172a83f1ea)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:51.764008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad819df-b397-465e-b15b-7c873ca9db1a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:51.766110 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2fd7d-3eb9-40af-bdda-d51399189be2 service swift] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:51.768868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1cf2fd7d-3eb9-40af-bdda-d51399189be2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:51.811982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2fd7d-3eb9-40af-bdda-d51399189be2 service swift] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:51.835326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2811618-0a87-43b4-a7bb-9e785e88098f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:51.836804 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2811618-0a87-43b4-a7bb-9e785e88098f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:51.837641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2811618-0a87-43b4-a7bb-9e785e88098f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:51.838955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2811618-0a87-43b4-a7bb-9e785e88098f admin admin] PATH_INFO: `/v3/projects/1a5857905b30465f8337fde981b8e02e` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:51.840940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2811618-0a87-43b4-a7bb-9e785e88098f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5857905b30465f8337fde981b8e02e)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:51.851662 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2811618-0a87-43b4-a7bb-9e785e88098f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:51.873816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3ad819df-b397-465e-b15b-7c873ca9db1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deaad1ec377f4786945c82172a83f1ea'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:51.891807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3ad819df-b397-465e-b15b-7c873ca9db1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deaad1ec377f4786945c82172a83f1ea'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:51.892981 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3ad819df-b397-465e-b15b-7c873ca9db1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deaad1ec377f4786945c82172a83f1ea'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:51.915935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3ad819df-b397-465e-b15b-7c873ca9db1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deaad1ec377f4786945c82172a83f1ea'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:51.916965 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3ad819df-b397-465e-b15b-7c873ca9db1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deaad1ec377f4786945c82172a83f1ea'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:51.942920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1287/2578] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:51 2022] DELETE /identity/v3/users/deaad1ec377f4786945c82172a83f1ea => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:51.944670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685d3ae-bc36-40b4-a5cd-6fefe626acba None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:51.945141 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685d3ae-bc36-40b4-a5cd-6fefe626acba None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:51.945622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685d3ae-bc36-40b4-a5cd-6fefe626acba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:51.963218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b2811618-0a87-43b4-a7bb-9e785e88098f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5857905b30465f8337fde981b8e02e', 'request_id': 'req-b2811618-0a87-43b4-a7bb-9e785e88098f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:51.970546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-0685d3ae-bc36-40b4-a5cd-6fefe626acba None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:51.972094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0685d3ae-bc36-40b4-a5cd-6fefe626acba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:51.976551 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b2811618-0a87-43b4-a7bb-9e785e88098f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5857905b30465f8337fde981b8e02e', 'request_id': 'req-b2811618-0a87-43b4-a7bb-9e785e88098f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:51.977792 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:51.989081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b2811618-0a87-43b4-a7bb-9e785e88098f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5857905b30465f8337fde981b8e02e', 'request_id': 'req-b2811618-0a87-43b4-a7bb-9e785e88098f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:52.011301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b2811618-0a87-43b4-a7bb-9e785e88098f admin admin] The token cache is being invalidate because project 1a5857905b30465f8337fde981b8e02e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:52.011301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b2811618-0a87-43b4-a7bb-9e785e88098f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:52.012175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b2811618-0a87-43b4-a7bb-9e785e88098f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:52.015188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1292/2579] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:51 2022] DELETE /identity/v3/projects/1a5857905b30465f8337fde981b8e02e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:55:52.024592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb71168-288d-4daf-a6d3-4f19c4667c71 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:52.025225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb71168-288d-4daf-a6d3-4f19c4667c71 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:52.025825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb71168-288d-4daf-a6d3-4f19c4667c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:52.062161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-5fb71168-288d-4daf-a6d3-4f19c4667c71 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:52.063712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-5fb71168-288d-4daf-a6d3-4f19c4667c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:52.083210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1288/2580] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:52.092171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d96d9b-b7ae-4df9-8813-2b06b751d7a0 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:52.093295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d96d9b-b7ae-4df9-8813-2b06b751d7a0 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:52.093655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d96d9b-b7ae-4df9-8813-2b06b751d7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:52.143431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f3d96d9b-b7ae-4df9-8813-2b06b751d7a0 None None] MFA Rules not processed for user `488e41d766134f3aa7c956816d01df06`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:52.145087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f3d96d9b-b7ae-4df9-8813-2b06b751d7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:52.196645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1293/2581] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:52.196645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb71168-288d-4daf-a6d3-4f19c4667c71 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:52.196645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-5fb71168-288d-4daf-a6d3-4f19c4667c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:52.245614 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1289/2582] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:52.246804 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f65af-1fbf-41c1-b0fa-3fa5f21abe40 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:52.247258 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f65af-1fbf-41c1-b0fa-3fa5f21abe40 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:52.247721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f65af-1fbf-41c1-b0fa-3fa5f21abe40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:52.272542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-881f65af-1fbf-41c1-b0fa-3fa5f21abe40 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:52.274128 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-881f65af-1fbf-41c1-b0fa-3fa5f21abe40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:52.277251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:52.286250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1290/2583] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:52.288410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f65af-1fbf-41c1-b0fa-3fa5f21abe40 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:52.290311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-881f65af-1fbf-41c1-b0fa-3fa5f21abe40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:52.303568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb71168-288d-4daf-a6d3-4f19c4667c71 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:52.307444 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cbfbc-451d-4ab0-a4d4-9542432b1973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:52.308065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cbfbc-451d-4ab0-a4d4-9542432b1973 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:52.308343 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cbfbc-451d-4ab0-a4d4-9542432b1973 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:52.308610 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cbfbc-451d-4ab0-a4d4-9542432b1973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:52.326532 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cbfbc-451d-4ab0-a4d4-9542432b1973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:52.331662 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cbfbc-451d-4ab0-a4d4-9542432b1973 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:52.354742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1294/2584] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:52.356216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cbfbc-451d-4ab0-a4d4-9542432b1973 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:52.376208 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f65af-1fbf-41c1-b0fa-3fa5f21abe40 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:52.381915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f220901-a808-48ae-b214-3aeb314756b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:52.382176 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cbfbc-451d-4ab0-a4d4-9542432b1973 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:52.382695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f220901-a808-48ae-b214-3aeb314756b5 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:52.382973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f220901-a808-48ae-b214-3aeb314756b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:52.383272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f220901-a808-48ae-b214-3aeb314756b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:52.388332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435f54-6ad4-4d16-a702-a92f1ae2d05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:52.389306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435f54-6ad4-4d16-a702-a92f1ae2d05d admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:52.389751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435f54-6ad4-4d16-a702-a92f1ae2d05d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:52.390057 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435f54-6ad4-4d16-a702-a92f1ae2d05d admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:52.391581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435f54-6ad4-4d16-a702-a92f1ae2d05d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:52.392090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435f54-6ad4-4d16-a702-a92f1ae2d05d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:52.393107 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f220901-a808-48ae-b214-3aeb314756b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:52.394102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435f54-6ad4-4d16-a702-a92f1ae2d05d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:52.394726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f220901-a808-48ae-b214-3aeb314756b5 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:52.407944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1291/2585] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:52.414518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1295/2586] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:52.421656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f220901-a808-48ae-b214-3aeb314756b5 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:52.437063 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f220901-a808-48ae-b214-3aeb314756b5 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:52.442349 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e984b-dc4b-48db-9944-847a95815a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:52.442963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e984b-dc4b-48db-9944-847a95815a8c admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:52.443997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e984b-dc4b-48db-9944-847a95815a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:52.444286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e984b-dc4b-48db-9944-847a95815a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:52.444945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e984b-dc4b-48db-9944-847a95815a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:52.447248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e984b-dc4b-48db-9944-847a95815a8c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:52.482828 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1292/2587] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:52 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:52.487891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435f54-6ad4-4d16-a702-a92f1ae2d05d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:52.499437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435f54-6ad4-4d16-a702-a92f1ae2d05d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:52.594356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7a70f-1b4d-4ec3-b075-12cdc28238d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:52.595166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7a70f-1b4d-4ec3-b075-12cdc28238d5 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:52.595510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7a70f-1b4d-4ec3-b075-12cdc28238d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:52.595920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7a70f-1b4d-4ec3-b075-12cdc28238d5 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:52.600147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7a70f-1b4d-4ec3-b075-12cdc28238d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:52.600147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7a70f-1b4d-4ec3-b075-12cdc28238d5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:52.631887 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1296/2588] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:52 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:55:52.640602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e984b-dc4b-48db-9944-847a95815a8c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:52.672963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e984b-dc4b-48db-9944-847a95815a8c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:52.682909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d939d-8eba-4cbe-a007-e58d0c22aa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:52.683925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d939d-8eba-4cbe-a007-e58d0c22aa28 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:52.684490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d939d-8eba-4cbe-a007-e58d0c22aa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:52.685067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d939d-8eba-4cbe-a007-e58d0c22aa28 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:52.686312 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d939d-8eba-4cbe-a007-e58d0c22aa28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:52.689380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d939d-8eba-4cbe-a007-e58d0c22aa28 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:52.707096 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1293/2589] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:52.712899 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7a70f-1b4d-4ec3-b075-12cdc28238d5 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:52.725631 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7a70f-1b4d-4ec3-b075-12cdc28238d5 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:52.731649 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae134ab9-f0f3-49d5-9c4d-794161dc667a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:52.732343 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae134ab9-f0f3-49d5-9c4d-794161dc667a admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:52.732689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae134ab9-f0f3-49d5-9c4d-794161dc667a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:52.733049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae134ab9-f0f3-49d5-9c4d-794161dc667a admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:52.734289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae134ab9-f0f3-49d5-9c4d-794161dc667a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:52.737740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae134ab9-f0f3-49d5-9c4d-794161dc667a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:52.755640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1297/2590] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:55:52.757697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d939d-8eba-4cbe-a007-e58d0c22aa28 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:52.775899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:52.790763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d939d-8eba-4cbe-a007-e58d0c22aa28 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:52.811638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f02935-cb60-4bbe-9693-4b03fd248448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:52.813145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02935-cb60-4bbe-9693-4b03fd248448 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:52.813878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02935-cb60-4bbe-9693-4b03fd248448 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:52.814347 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02935-cb60-4bbe-9693-4b03fd248448 admin admin] PATH_INFO: `/v3/projects/dff32c14998645219b45dcf04be45d54/users/6aab80af35e346158bacd7737b72b862/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:52.845169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f02935-cb60-4bbe-9693-4b03fd248448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff32c14998645219b45dcf04be45d54, user_id=6aab80af35e346158bacd7737b72b862, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:52.849246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f02935-cb60-4bbe-9693-4b03fd248448 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:52.876926 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1294/2591] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:52 2022] PUT /identity/v3/projects/dff32c14998645219b45dcf04be45d54/users/6aab80af35e346158bacd7737b72b862/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:52.884956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae134ab9-f0f3-49d5-9c4d-794161dc667a admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:52.887566 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ae134ab9-f0f3-49d5-9c4d-794161dc667a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:52.897041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e8047-3d22-40d0-b524-45637bca8765 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:52.897538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e8047-3d22-40d0-b524-45637bca8765 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:52.898027 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e8047-3d22-40d0-b524-45637bca8765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:52.925893 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae134ab9-f0f3-49d5-9c4d-794161dc667a admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:52.948283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-c82e8047-3d22-40d0-b524-45637bca8765 None None] MFA Rules not processed for user `6aab80af35e346158bacd7737b72b862`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:52.949604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c82e8047-3d22-40d0-b524-45637bca8765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:53.011402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:53.012053 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:53.012443 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:53.012727 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] PATH_INFO: `/v3/projects/9d33a1d09fa54fc7b852597772c78751` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:53.014162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d33a1d09fa54fc7b852597772c78751)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:53.016130 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:53.051356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1295/2592] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:53.063445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e8047-3d22-40d0-b524-45637bca8765 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:53.080101 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d33a1d09fa54fc7b852597772c78751', 'request_id': 'req-cfe43026-39b7-486d-b410-e6c52e92d740'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.082024 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:53.091731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d33a1d09fa54fc7b852597772c78751', 'request_id': 'req-cfe43026-39b7-486d-b410-e6c52e92d740'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.101597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d33a1d09fa54fc7b852597772c78751', 'request_id': 'req-cfe43026-39b7-486d-b410-e6c52e92d740'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.114417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] The token cache is being invalidate because project 9d33a1d09fa54fc7b852597772c78751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:53.114811 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.116358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.118006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1298/2593] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:52 2022] DELETE /identity/v3/projects/9d33a1d09fa54fc7b852597772c78751 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:53.122173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:53.123648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:53.133166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e8047-3d22-40d0-b524-45637bca8765 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:53.196364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd91603-559c-45ae-9001-6caa27daa905 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:53.197198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd91603-559c-45ae-9001-6caa27daa905 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:53.197590 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd91603-559c-45ae-9001-6caa27daa905 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:53.197982 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd91603-559c-45ae-9001-6caa27daa905 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:53.200305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0cd91603-559c-45ae-9001-6caa27daa905 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:53.203197 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe43026-39b7-486d-b410-e6c52e92d740 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:53.208326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7d2bf-b2eb-4f4b-823c-b1ece0b12ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:53.210350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d2bf-b2eb-4f4b-823c-b1ece0b12ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:53.210644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d2bf-b2eb-4f4b-823c-b1ece0b12ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:53.210914 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d2bf-b2eb-4f4b-823c-b1ece0b12ad8 admin admin] PATH_INFO: `/v3/users/35a32b264db74b33ae621b886bdef7e8` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:53.212235 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7d2bf-b2eb-4f4b-823c-b1ece0b12ad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35a32b264db74b33ae621b886bdef7e8)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:53.214140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7d2bf-b2eb-4f4b-823c-b1ece0b12ad8 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:53.248161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:53.272941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-25c7d2bf-b2eb-4f4b-823c-b1ece0b12ad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35a32b264db74b33ae621b886bdef7e8'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.273354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd91603-559c-45ae-9001-6caa27daa905 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:53.274865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd91603-559c-45ae-9001-6caa27daa905 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:53.285162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1296/2594] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:55:53.287643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-25c7d2bf-b2eb-4f4b-823c-b1ece0b12ad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35a32b264db74b33ae621b886bdef7e8'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.288428 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-25c7d2bf-b2eb-4f4b-823c-b1ece0b12ad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35a32b264db74b33ae621b886bdef7e8'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.289046 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-25c7d2bf-b2eb-4f4b-823c-b1ece0b12ad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35a32b264db74b33ae621b886bdef7e8'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.298204 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-25c7d2bf-b2eb-4f4b-823c-b1ece0b12ad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35a32b264db74b33ae621b886bdef7e8'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.312883 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1299/2595] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:53 2022] DELETE /identity/v3/users/35a32b264db74b33ae621b886bdef7e8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:55:53.622275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd91603-559c-45ae-9001-6caa27daa905 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:53.635669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd91603-559c-45ae-9001-6caa27daa905 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:53.698077 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dc143-1fed-4549-9c48-b7a89ebfe3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:53.698696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dc143-1fed-4549-9c48-b7a89ebfe3dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:53.699029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dc143-1fed-4549-9c48-b7a89ebfe3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:53.699397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dc143-1fed-4549-9c48-b7a89ebfe3dc admin admin] PATH_INFO: `/v3/projects/b5ec60c5a3e34ff78c94b7ef06adcc12` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:53.700495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dc143-1fed-4549-9c48-b7a89ebfe3dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ec60c5a3e34ff78c94b7ef06adcc12)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:53.702384 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dc143-1fed-4549-9c48-b7a89ebfe3dc admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:53.749708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-746dc143-1fed-4549-9c48-b7a89ebfe3dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ec60c5a3e34ff78c94b7ef06adcc12', 'request_id': 'req-746dc143-1fed-4549-9c48-b7a89ebfe3dc'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.756346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-746dc143-1fed-4549-9c48-b7a89ebfe3dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ec60c5a3e34ff78c94b7ef06adcc12', 'request_id': 'req-746dc143-1fed-4549-9c48-b7a89ebfe3dc'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.761911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-746dc143-1fed-4549-9c48-b7a89ebfe3dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ec60c5a3e34ff78c94b7ef06adcc12', 'request_id': 'req-746dc143-1fed-4549-9c48-b7a89ebfe3dc'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.774265 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-746dc143-1fed-4549-9c48-b7a89ebfe3dc admin admin] The token cache is being invalidate because project b5ec60c5a3e34ff78c94b7ef06adcc12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:53.774557 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-746dc143-1fed-4549-9c48-b7a89ebfe3dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.775246 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-746dc143-1fed-4549-9c48-b7a89ebfe3dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:53.776777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1297/2596] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:53 2022] DELETE /identity/v3/projects/b5ec60c5a3e34ff78c94b7ef06adcc12 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:53.781807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd5a56-63cc-4f04-841f-04697134b7c5 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:53.782090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd5a56-63cc-4f04-841f-04697134b7c5 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:53.782374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd5a56-63cc-4f04-841f-04697134b7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:53.802301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-edcd5a56-63cc-4f04-841f-04697134b7c5 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:53.804381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-edcd5a56-63cc-4f04-841f-04697134b7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:53.855462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:53.869071 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1300/2597] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:53.877750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091044b6-4b50-493f-8b5d-ae83f03c3cb2 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:53.878059 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091044b6-4b50-493f-8b5d-ae83f03c3cb2 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:53.878334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091044b6-4b50-493f-8b5d-ae83f03c3cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:53.899794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-091044b6-4b50-493f-8b5d-ae83f03c3cb2 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:53.901048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-091044b6-4b50-493f-8b5d-ae83f03c3cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:53.909539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1298/2598] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:53.914761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd5a56-63cc-4f04-841f-04697134b7c5 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:53.920457 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962ca42-90cc-4064-9140-d3cc18c7df8c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:53.920744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962ca42-90cc-4064-9140-d3cc18c7df8c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:53.921018 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962ca42-90cc-4064-9140-d3cc18c7df8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:53.925435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd5a56-63cc-4f04-841f-04697134b7c5 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:53.930113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5de87-4604-4be0-845d-97b4c1a87db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:53.930719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5de87-4604-4be0-845d-97b4c1a87db6 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:53.930999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5de87-4604-4be0-845d-97b4c1a87db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:53.931309 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5de87-4604-4be0-845d-97b4c1a87db6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:53.931989 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5de87-4604-4be0-845d-97b4c1a87db6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:53.932289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5de87-4604-4be0-845d-97b4c1a87db6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:53.933887 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5de87-4604-4be0-845d-97b4c1a87db6 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:53.940719 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-0962ca42-90cc-4064-9140-d3cc18c7df8c None None] MFA Rules not processed for user `488e41d766134f3aa7c956816d01df06`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:53.941923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0962ca42-90cc-4064-9140-d3cc18c7df8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:53.946672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1301/2599] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:53.951773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5de87-4604-4be0-845d-97b4c1a87db6 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:53.962700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5de87-4604-4be0-845d-97b4c1a87db6 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:53.967673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171a606-b41a-4557-b1cf-322282938572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:53.968272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171a606-b41a-4557-b1cf-322282938572 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:53.968553 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171a606-b41a-4557-b1cf-322282938572 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:53.968819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171a606-b41a-4557-b1cf-322282938572 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:53.969502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171a606-b41a-4557-b1cf-322282938572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:53.971508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171a606-b41a-4557-b1cf-322282938572 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:54.005481 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1302/2600] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:53 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:55:54.007284 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1299/2601] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:54.010196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962ca42-90cc-4064-9140-d3cc18c7df8c None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:54.023035 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962ca42-90cc-4064-9140-d3cc18c7df8c None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:54.028585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:54.102206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5216d7-69a0-41a4-8d0a-dd0841e57d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:54.102857 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5216d7-69a0-41a4-8d0a-dd0841e57d02 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:54.103310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5216d7-69a0-41a4-8d0a-dd0841e57d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:54.103588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5216d7-69a0-41a4-8d0a-dd0841e57d02 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:54.104218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5216d7-69a0-41a4-8d0a-dd0841e57d02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:54.106254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5216d7-69a0-41a4-8d0a-dd0841e57d02 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:54.130933 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1300/2602] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:54 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:54.136522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171a606-b41a-4557-b1cf-322282938572 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:54.149779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171a606-b41a-4557-b1cf-322282938572 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:54.155584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243cd07-dcda-4d6b-9c99-e7e92e362af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:54.156251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243cd07-dcda-4d6b-9c99-e7e92e362af8 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:54.156486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243cd07-dcda-4d6b-9c99-e7e92e362af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:54.156766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243cd07-dcda-4d6b-9c99-e7e92e362af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:54.157424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243cd07-dcda-4d6b-9c99-e7e92e362af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:54.159040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243cd07-dcda-4d6b-9c99-e7e92e362af8 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:54.172854 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1303/2603] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:54.177098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5216d7-69a0-41a4-8d0a-dd0841e57d02 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:54.187945 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5216d7-69a0-41a4-8d0a-dd0841e57d02 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:54.192778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f1c7d-1ee6-4f3b-9562-1971d604ef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:54.193436 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f1c7d-1ee6-4f3b-9562-1971d604ef37 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:54.194048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f1c7d-1ee6-4f3b-9562-1971d604ef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:54.194153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f1c7d-1ee6-4f3b-9562-1971d604ef37 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:54.194761 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f1c7d-1ee6-4f3b-9562-1971d604ef37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:54.196440 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f1c7d-1ee6-4f3b-9562-1971d604ef37 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:54.206930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1301/2604] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:54.212508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243cd07-dcda-4d6b-9c99-e7e92e362af8 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:54.223022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243cd07-dcda-4d6b-9c99-e7e92e362af8 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:54.228649 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd671ec-b10b-4eb5-bfae-570dd27322d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:54.229327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd671ec-b10b-4eb5-bfae-570dd27322d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:54.229601 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd671ec-b10b-4eb5-bfae-570dd27322d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:54.229871 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd671ec-b10b-4eb5-bfae-570dd27322d4 admin admin] PATH_INFO: `/v3/projects/289a5a1d9cf04e5cb7ef9f097dc22b97/users/ab4ffaef97814ddcbb3e91cc609d9340/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:54.250665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd671ec-b10b-4eb5-bfae-570dd27322d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289a5a1d9cf04e5cb7ef9f097dc22b97, user_id=ab4ffaef97814ddcbb3e91cc609d9340, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:54.253930 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd671ec-b10b-4eb5-bfae-570dd27322d4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:54.275182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1304/2605] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:54 2022] PUT /identity/v3/projects/289a5a1d9cf04e5cb7ef9f097dc22b97/users/ab4ffaef97814ddcbb3e91cc609d9340/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:54.289745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703b532-f061-4094-b576-6b1cd7884b4d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:54.290029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703b532-f061-4094-b576-6b1cd7884b4d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:54.290332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703b532-f061-4094-b576-6b1cd7884b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:54.322357 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-7703b532-f061-4094-b576-6b1cd7884b4d None None] MFA Rules not processed for user `ab4ffaef97814ddcbb3e91cc609d9340`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:54.323613 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7703b532-f061-4094-b576-6b1cd7884b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:54.392293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1302/2606] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:54.397450 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd671ec-b10b-4eb5-bfae-570dd27322d4 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:54.465353 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd671ec-b10b-4eb5-bfae-570dd27322d4 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:54.470332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1dc78-ce0f-4f13-8657-18166254cfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:54.470939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1dc78-ce0f-4f13-8657-18166254cfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:54.471247 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1dc78-ce0f-4f13-8657-18166254cfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:54.471525 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1dc78-ce0f-4f13-8657-18166254cfc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:54.472173 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1dc78-ce0f-4f13-8657-18166254cfc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:54.474069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1dc78-ce0f-4f13-8657-18166254cfc5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:54.501954 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1305/2607] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:54 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:54.506502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703b532-f061-4094-b576-6b1cd7884b4d None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:54.518142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703b532-f061-4094-b576-6b1cd7884b4d None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:54.587172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347cbbe-ee65-4443-b13f-aa6610236ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:54.587933 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347cbbe-ee65-4443-b13f-aa6610236ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:54.588343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347cbbe-ee65-4443-b13f-aa6610236ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:54.588633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347cbbe-ee65-4443-b13f-aa6610236ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:54.589273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347cbbe-ee65-4443-b13f-aa6610236ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:54.591369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347cbbe-ee65-4443-b13f-aa6610236ae2 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:54.625190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1303/2608] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:54 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:54.629150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1dc78-ce0f-4f13-8657-18166254cfc5 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:54.643855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1dc78-ce0f-4f13-8657-18166254cfc5 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:54.651517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84479170-6083-4352-8a23-6bb1f6fdfc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:54.652272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84479170-6083-4352-8a23-6bb1f6fdfc22 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:54.652659 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84479170-6083-4352-8a23-6bb1f6fdfc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:54.667977 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84479170-6083-4352-8a23-6bb1f6fdfc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:54.668960 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84479170-6083-4352-8a23-6bb1f6fdfc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:54.671207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84479170-6083-4352-8a23-6bb1f6fdfc22 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:54.689776 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1306/2609] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:55:54.691294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347cbbe-ee65-4443-b13f-aa6610236ae2 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:54.697980 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:54.703490 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347cbbe-ee65-4443-b13f-aa6610236ae2 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:54.719691 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f77ab7-6078-4283-b854-e0d645018b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:54.720472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f77ab7-6078-4283-b854-e0d645018b24 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:54.720751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f77ab7-6078-4283-b854-e0d645018b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:54.721025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f77ab7-6078-4283-b854-e0d645018b24 admin admin] PATH_INFO: `/v3/projects/093c5e255162435cb63ad6b821879e10/users/46cc8ed0e45e4ecc8c39b3baba753c80/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:54.743751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f77ab7-6078-4283-b854-e0d645018b24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=093c5e255162435cb63ad6b821879e10, user_id=46cc8ed0e45e4ecc8c39b3baba753c80, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:54.747381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f77ab7-6078-4283-b854-e0d645018b24 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:54.763991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1304/2610] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:54 2022] PUT /identity/v3/projects/093c5e255162435cb63ad6b821879e10/users/46cc8ed0e45e4ecc8c39b3baba753c80/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:54.767879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84479170-6083-4352-8a23-6bb1f6fdfc22 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:54.779578 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84479170-6083-4352-8a23-6bb1f6fdfc22 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:54.858537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ca2df-55af-4b31-ae2a-93bbebdb61ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:54.859183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ca2df-55af-4b31-ae2a-93bbebdb61ec admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:54.859470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ca2df-55af-4b31-ae2a-93bbebdb61ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:54.859891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ca2df-55af-4b31-ae2a-93bbebdb61ec admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:54.860652 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ca2df-55af-4b31-ae2a-93bbebdb61ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:54.862322 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ca2df-55af-4b31-ae2a-93bbebdb61ec admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:54.865387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:54.876502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1307/2611] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:54.880588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f77ab7-6078-4283-b854-e0d645018b24 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:54.893847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f77ab7-6078-4283-b854-e0d645018b24 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:54.901327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2cad2-aec7-46ef-9a91-698f1c05822e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:54.902111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2cad2-aec7-46ef-9a91-698f1c05822e admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:54.902463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2cad2-aec7-46ef-9a91-698f1c05822e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:54.902803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2cad2-aec7-46ef-9a91-698f1c05822e admin admin] PATH_INFO: `/v3/projects/093c5e255162435cb63ad6b821879e10/users/46cc8ed0e45e4ecc8c39b3baba753c80/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:54.906600 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2cad2-aec7-46ef-9a91-698f1c05822e admin admin] RBAC: Authorizing `identity:create_grant(project_id=093c5e255162435cb63ad6b821879e10, user_id=46cc8ed0e45e4ecc8c39b3baba753c80, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:54.910633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2cad2-aec7-46ef-9a91-698f1c05822e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:54.925554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1305/2612] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:54 2022] PUT /identity/v3/projects/093c5e255162435cb63ad6b821879e10/users/46cc8ed0e45e4ecc8c39b3baba753c80/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:54.936409 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50177ad-3241-4827-9ad1-25ef197aa53d None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:54.936769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50177ad-3241-4827-9ad1-25ef197aa53d None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:54.937116 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50177ad-3241-4827-9ad1-25ef197aa53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:54.974126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-a50177ad-3241-4827-9ad1-25ef197aa53d None None] MFA Rules not processed for user `46cc8ed0e45e4ecc8c39b3baba753c80`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:54.976352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a50177ad-3241-4827-9ad1-25ef197aa53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:55.076869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1308/2613] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:55.090041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2cad2-aec7-46ef-9a91-698f1c05822e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:55.093453 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-add2cad2-aec7-46ef-9a91-698f1c05822e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:55.175042 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2cad2-aec7-46ef-9a91-698f1c05822e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:55.179848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955f350-b824-47b0-9012-a73721f2ad93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:55.180462 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955f350-b824-47b0-9012-a73721f2ad93 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:55.180910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955f350-b824-47b0-9012-a73721f2ad93 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:55.181188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955f350-b824-47b0-9012-a73721f2ad93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:55.194826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955f350-b824-47b0-9012-a73721f2ad93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:55.194826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955f350-b824-47b0-9012-a73721f2ad93 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:55.209860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1306/2614] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:55:55.301559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50177ad-3241-4827-9ad1-25ef197aa53d None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:55.393015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50177ad-3241-4827-9ad1-25ef197aa53d None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:55.402658 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:55.404251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:55.404831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:55.405414 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] PATH_INFO: `/v3/users/ab4ffaef97814ddcbb3e91cc609d9340` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:55.407978 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab4ffaef97814ddcbb3e91cc609d9340)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:55.413930 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:55.512420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab4ffaef97814ddcbb3e91cc609d9340'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:55.531021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab4ffaef97814ddcbb3e91cc609d9340'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:55.533016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab4ffaef97814ddcbb3e91cc609d9340'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:55.533797 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab4ffaef97814ddcbb3e91cc609d9340'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:55.543219 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab4ffaef97814ddcbb3e91cc609d9340'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:55.557182 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1309/2615] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:55 2022] DELETE /identity/v3/users/ab4ffaef97814ddcbb3e91cc609d9340 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:55.561879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1a27b-1ccb-4cae-9c6d-5e8d215532be None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:55.562144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1a27b-1ccb-4cae-9c6d-5e8d215532be None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:55.562556 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1a27b-1ccb-4cae-9c6d-5e8d215532be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:55.584785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-93e1a27b-1ccb-4cae-9c6d-5e8d215532be None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:55.585953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-93e1a27b-1ccb-4cae-9c6d-5e8d215532be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:55.652883 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1307/2616] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:55.663868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:55.665427 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:55.671217 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:55.753619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725a6d9-b290-4c47-849c-56806c1edc13 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:55.758922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4f9ef-7911-4b9a-afde-c6010591dace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:55.759643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4f9ef-7911-4b9a-afde-c6010591dace service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:55.759922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4f9ef-7911-4b9a-afde-c6010591dace service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:55.760220 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4f9ef-7911-4b9a-afde-c6010591dace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:55.775320 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4f9ef-7911-4b9a-afde-c6010591dace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:55.777954 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4f9ef-7911-4b9a-afde-c6010591dace service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:55.792795 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1310/2617] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:56.239294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1a27b-1ccb-4cae-9c6d-5e8d215532be None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:56.241461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-93e1a27b-1ccb-4cae-9c6d-5e8d215532be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:56.259864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1a27b-1ccb-4cae-9c6d-5e8d215532be None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:56.264860 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:56.265548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:56.265841 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:56.266194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] PATH_INFO: `/v3/projects/289a5a1d9cf04e5cb7ef9f097dc22b97` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:56.267338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=289a5a1d9cf04e5cb7ef9f097dc22b97)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:56.269787 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:56.320549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '289a5a1d9cf04e5cb7ef9f097dc22b97', 'request_id': 'req-1e75db2c-e551-49f1-a740-8f0d8fc844fc'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.327082 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '289a5a1d9cf04e5cb7ef9f097dc22b97', 'request_id': 'req-1e75db2c-e551-49f1-a740-8f0d8fc844fc'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.333372 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '289a5a1d9cf04e5cb7ef9f097dc22b97', 'request_id': 'req-1e75db2c-e551-49f1-a740-8f0d8fc844fc'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.336787 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:56.349363 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] The token cache is being invalidate because project 289a5a1d9cf04e5cb7ef9f097dc22b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:56.349896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.350657 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.352316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1308/2618] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:56 2022] DELETE /identity/v3/projects/289a5a1d9cf04e5cb7ef9f097dc22b97 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:56.356568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4f9ef-7911-4b9a-afde-c6010591dace service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:56.369467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4f9ef-7911-4b9a-afde-c6010591dace service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:56.439635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323462d-fb12-4046-b357-ca45dbebf255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:56.440341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323462d-fb12-4046-b357-ca45dbebf255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:56.440625 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323462d-fb12-4046-b357-ca45dbebf255 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:56.441000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323462d-fb12-4046-b357-ca45dbebf255 admin admin] PATH_INFO: `/v3/users/46cc8ed0e45e4ecc8c39b3baba753c80` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:56.442098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323462d-fb12-4046-b357-ca45dbebf255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46cc8ed0e45e4ecc8c39b3baba753c80)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:56.444918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323462d-fb12-4046-b357-ca45dbebf255 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:56.509191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:56.510699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:56.512269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7323462d-fb12-4046-b357-ca45dbebf255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46cc8ed0e45e4ecc8c39b3baba753c80'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.524745 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75db2c-e551-49f1-a740-8f0d8fc844fc admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:56.528503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7323462d-fb12-4046-b357-ca45dbebf255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46cc8ed0e45e4ecc8c39b3baba753c80'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.529222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7323462d-fb12-4046-b357-ca45dbebf255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46cc8ed0e45e4ecc8c39b3baba753c80'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.529894 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7323462d-fb12-4046-b357-ca45dbebf255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46cc8ed0e45e4ecc8c39b3baba753c80'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.530616 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:56.531438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:56.531777 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:56.532033 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] PATH_INFO: `/v3/users/6aab80af35e346158bacd7737b72b862` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:56.533259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aab80af35e346158bacd7737b72b862)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:56.535471 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:56.541093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-7323462d-fb12-4046-b357-ca45dbebf255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46cc8ed0e45e4ecc8c39b3baba753c80'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.569285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1311/2619] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:56 2022] DELETE /identity/v3/users/46cc8ed0e45e4ecc8c39b3baba753c80 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:56.606596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aab80af35e346158bacd7737b72b862'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.618276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aab80af35e346158bacd7737b72b862'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.619288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aab80af35e346158bacd7737b72b862'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.638314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aab80af35e346158bacd7737b72b862'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.639094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aab80af35e346158bacd7737b72b862'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:56.664252 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1309/2620] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:56 2022] DELETE /identity/v3/users/6aab80af35e346158bacd7737b72b862 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:55:56.669889 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a6c0d-6239-4ce5-b536-c2bd89ef970c None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:56.670356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a6c0d-6239-4ce5-b536-c2bd89ef970c None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:56.670802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a6c0d-6239-4ce5-b536-c2bd89ef970c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:56.698435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-7c4a6c0d-6239-4ce5-b536-c2bd89ef970c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:56.699802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7c4a6c0d-6239-4ce5-b536-c2bd89ef970c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:56.728617 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:56.785939 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1312/2621] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:56.798710 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:56.801762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:56.904579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e14a3-b032-4d0e-b58a-3ecb082f857a admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:56.911598 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb4c75-7f53-4d5c-9584-e44927f09470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:56.912823 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb4c75-7f53-4d5c-9584-e44927f09470 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:56.913287 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb4c75-7f53-4d5c-9584-e44927f09470 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:56.913732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb4c75-7f53-4d5c-9584-e44927f09470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:56.941924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb4c75-7f53-4d5c-9584-e44927f09470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:56.946248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb4c75-7f53-4d5c-9584-e44927f09470 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:56.967019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1310/2622] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:57.155065 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a6c0d-6239-4ce5-b536-c2bd89ef970c None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:57.156603 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7c4a6c0d-6239-4ce5-b536-c2bd89ef970c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:57.170923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a6c0d-6239-4ce5-b536-c2bd89ef970c None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:57.176980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603cc6a-41dc-481b-9f94-212590fbd7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:57.178850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603cc6a-41dc-481b-9f94-212590fbd7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.179606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603cc6a-41dc-481b-9f94-212590fbd7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.179882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603cc6a-41dc-481b-9f94-212590fbd7c8 admin admin] PATH_INFO: `/v3/projects/093c5e255162435cb63ad6b821879e10` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.181098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603cc6a-41dc-481b-9f94-212590fbd7c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=093c5e255162435cb63ad6b821879e10)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:57.183024 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603cc6a-41dc-481b-9f94-212590fbd7c8 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:57.247815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-6603cc6a-41dc-481b-9f94-212590fbd7c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '093c5e255162435cb63ad6b821879e10', 'request_id': 'req-6603cc6a-41dc-481b-9f94-212590fbd7c8'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:57.257686 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-6603cc6a-41dc-481b-9f94-212590fbd7c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '093c5e255162435cb63ad6b821879e10', 'request_id': 'req-6603cc6a-41dc-481b-9f94-212590fbd7c8'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:57.264822 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-6603cc6a-41dc-481b-9f94-212590fbd7c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '093c5e255162435cb63ad6b821879e10', 'request_id': 'req-6603cc6a-41dc-481b-9f94-212590fbd7c8'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:57.276454 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-6603cc6a-41dc-481b-9f94-212590fbd7c8 admin admin] The token cache is being invalidate because project 093c5e255162435cb63ad6b821879e10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:57.276743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-6603cc6a-41dc-481b-9f94-212590fbd7c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:57.277533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-6603cc6a-41dc-481b-9f94-212590fbd7c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:57.279040 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1313/2623] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:57 2022] DELETE /identity/v3/projects/093c5e255162435cb63ad6b821879e10 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:57.283940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740749a0-2f2b-46df-8ea8-a314061a3f11 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.284222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740749a0-2f2b-46df-8ea8-a314061a3f11 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.284492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740749a0-2f2b-46df-8ea8-a314061a3f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.304048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-740749a0-2f2b-46df-8ea8-a314061a3f11 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:57.305270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-740749a0-2f2b-46df-8ea8-a314061a3f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:57.370931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1311/2624] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:57.379676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29712dc7-5399-4765-8af1-c4e2a2aa8039 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.380306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29712dc7-5399-4765-8af1-c4e2a2aa8039 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.381978 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29712dc7-5399-4765-8af1-c4e2a2aa8039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.416878 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-29712dc7-5399-4765-8af1-c4e2a2aa8039 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:57.419031 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-29712dc7-5399-4765-8af1-c4e2a2aa8039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:57.433598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1314/2625] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:57.439573 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740749a0-2f2b-46df-8ea8-a314061a3f11 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:57.451677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740749a0-2f2b-46df-8ea8-a314061a3f11 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:57.456361 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29757eae-839e-4af1-877e-dfb7bf41dafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:57.457064 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29757eae-839e-4af1-877e-dfb7bf41dafd admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.457414 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29757eae-839e-4af1-877e-dfb7bf41dafd admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.457747 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29757eae-839e-4af1-877e-dfb7bf41dafd admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.458540 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29757eae-839e-4af1-877e-dfb7bf41dafd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:57.459410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29757eae-839e-4af1-877e-dfb7bf41dafd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:57.461153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29757eae-839e-4af1-877e-dfb7bf41dafd admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:57.464602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:57.480807 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1312/2626] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:57.481935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29712dc7-5399-4765-8af1-c4e2a2aa8039 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:57.491577 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29712dc7-5399-4765-8af1-c4e2a2aa8039 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:57.496214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4d82a-3f0b-4c67-951c-a5c102c0e11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:57.496819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4d82a-3f0b-4c67-951c-a5c102c0e11c admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.497097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4d82a-3f0b-4c67-951c-a5c102c0e11c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.497846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4d82a-3f0b-4c67-951c-a5c102c0e11c admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.498014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4d82a-3f0b-4c67-951c-a5c102c0e11c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:57.499957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4d82a-3f0b-4c67-951c-a5c102c0e11c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:57.530907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1315/2627] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:57 2022] POST /identity/v3/projects => generated 407 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:55:57.535341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29757eae-839e-4af1-877e-dfb7bf41dafd admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:57.546393 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29757eae-839e-4af1-877e-dfb7bf41dafd admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:57.602520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4d82a-3f0b-4c67-951c-a5c102c0e11c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:57.605081 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ebb4d82a-3f0b-4c67-951c-a5c102c0e11c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:57.617446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59371e46-f219-42ae-9cab-b0165669404d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:57.618055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59371e46-f219-42ae-9cab-b0165669404d admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.618333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59371e46-f219-42ae-9cab-b0165669404d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.618706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59371e46-f219-42ae-9cab-b0165669404d admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.619396 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59371e46-f219-42ae-9cab-b0165669404d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:57.621819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59371e46-f219-42ae-9cab-b0165669404d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:57.656858 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1313/2628] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:57 2022] POST /identity/v3/users => generated 304 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:57.657741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59371e46-f219-42ae-9cab-b0165669404d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:57.670310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59371e46-f219-42ae-9cab-b0165669404d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:57.675501 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447504b3-62eb-4249-a56c-9dc3920c5085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:57.676218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447504b3-62eb-4249-a56c-9dc3920c5085 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.676566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447504b3-62eb-4249-a56c-9dc3920c5085 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.676912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447504b3-62eb-4249-a56c-9dc3920c5085 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.678717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447504b3-62eb-4249-a56c-9dc3920c5085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:57.680514 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447504b3-62eb-4249-a56c-9dc3920c5085 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:57.682870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4d82a-3f0b-4c67-951c-a5c102c0e11c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:57.689134 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08187c-8e66-40c0-b35b-61081d0ce8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:57.689868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08187c-8e66-40c0-b35b-61081d0ce8b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.690218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08187c-8e66-40c0-b35b-61081d0ce8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.690559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08187c-8e66-40c0-b35b-61081d0ce8b7 admin admin] PATH_INFO: `/v3/projects/dff32c14998645219b45dcf04be45d54` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.691877 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08187c-8e66-40c0-b35b-61081d0ce8b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff32c14998645219b45dcf04be45d54)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:57.694518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08187c-8e66-40c0-b35b-61081d0ce8b7 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:57.697873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1314/2629] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:57.699111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447504b3-62eb-4249-a56c-9dc3920c5085 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:57.712309 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447504b3-62eb-4249-a56c-9dc3920c5085 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:57.718156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e25f63-43d7-4393-955f-6a45d67cabea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:57.719142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e25f63-43d7-4393-955f-6a45d67cabea admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.719293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e25f63-43d7-4393-955f-6a45d67cabea admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.719464 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e25f63-43d7-4393-955f-6a45d67cabea admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.720207 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e25f63-43d7-4393-955f-6a45d67cabea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:57.722062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e25f63-43d7-4393-955f-6a45d67cabea admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:57.725323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:57.739713 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1315/2630] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:57.741974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e25f63-43d7-4393-955f-6a45d67cabea admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:57.752565 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e25f63-43d7-4393-955f-6a45d67cabea admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:57.759149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f08187c-8e66-40c0-b35b-61081d0ce8b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff32c14998645219b45dcf04be45d54', 'request_id': 'req-2f08187c-8e66-40c0-b35b-61081d0ce8b7'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:57.768519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f08187c-8e66-40c0-b35b-61081d0ce8b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff32c14998645219b45dcf04be45d54', 'request_id': 'req-2f08187c-8e66-40c0-b35b-61081d0ce8b7'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:57.783490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f08187c-8e66-40c0-b35b-61081d0ce8b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff32c14998645219b45dcf04be45d54', 'request_id': 'req-2f08187c-8e66-40c0-b35b-61081d0ce8b7'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:57.816420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f08187c-8e66-40c0-b35b-61081d0ce8b7 admin admin] The token cache is being invalidate because project dff32c14998645219b45dcf04be45d54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:57.816813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f08187c-8e66-40c0-b35b-61081d0ce8b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:57.817538 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-2f08187c-8e66-40c0-b35b-61081d0ce8b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:57.819522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1316/2631] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:57 2022] DELETE /identity/v3/projects/dff32c14998645219b45dcf04be45d54 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:57.825439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b1dcf-9bc6-4c07-9974-5d533b457ec3 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.825718 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b1dcf-9bc6-4c07-9974-5d533b457ec3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.825992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b1dcf-9bc6-4c07-9974-5d533b457ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.831576 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23e97a-2237-4bee-a5ca-d1a49b005b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:57.832803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23e97a-2237-4bee-a5ca-d1a49b005b0e admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.833250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23e97a-2237-4bee-a5ca-d1a49b005b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.834686 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23e97a-2237-4bee-a5ca-d1a49b005b0e admin admin] PATH_INFO: `/v3/projects/e39f9909594c4a0d9e148ed3b443bde9/users/4dd59dfda97d40aaaa9706ab1029a352/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.854544 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-cb9b1dcf-9bc6-4c07-9974-5d533b457ec3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:57.854766 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23e97a-2237-4bee-a5ca-d1a49b005b0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39f9909594c4a0d9e148ed3b443bde9, user_id=4dd59dfda97d40aaaa9706ab1029a352, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:57.856666 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-cb9b1dcf-9bc6-4c07-9974-5d533b457ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:57.857866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23e97a-2237-4bee-a5ca-d1a49b005b0e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:57.870813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1317/2632] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:57.879906 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcd1b5-9acb-4f3e-96f3-f4766fdd8271 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.880166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcd1b5-9acb-4f3e-96f3-f4766fdd8271 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.880467 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcd1b5-9acb-4f3e-96f3-f4766fdd8271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.880713 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1316/2633] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:57 2022] PUT /identity/v3/projects/e39f9909594c4a0d9e148ed3b443bde9/users/4dd59dfda97d40aaaa9706ab1029a352/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:55:57.893329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0e86f-bfd5-4f4f-95e7-382f3ed5b5f8 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:57.893668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0e86f-bfd5-4f4f-95e7-382f3ed5b5f8 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:57.893876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0e86f-bfd5-4f4f-95e7-382f3ed5b5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:57.923406 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-96fcd1b5-9acb-4f3e-96f3-f4766fdd8271 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:57.925289 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-96fcd1b5-9acb-4f3e-96f3-f4766fdd8271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:57.931008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f4b0e86f-bfd5-4f4f-95e7-382f3ed5b5f8 None None] MFA Rules not processed for user `4dd59dfda97d40aaaa9706ab1029a352`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:57.932409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f4b0e86f-bfd5-4f4f-95e7-382f3ed5b5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:57.973219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:58.021685 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1318/2634] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:58.023326 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1317/2635] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:57 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:58.025386 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0e86f-bfd5-4f4f-95e7-382f3ed5b5f8 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.026592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcd1b5-9acb-4f3e-96f3-f4766fdd8271 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.029386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-96fcd1b5-9acb-4f3e-96f3-f4766fdd8271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:58.049983 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0e86f-bfd5-4f4f-95e7-382f3ed5b5f8 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.060015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89b0e1-9265-4d57-9a2e-d45fbc624969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.060015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89b0e1-9265-4d57-9a2e-d45fbc624969 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.060015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89b0e1-9265-4d57-9a2e-d45fbc624969 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.060015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89b0e1-9265-4d57-9a2e-d45fbc624969 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.060534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89b0e1-9265-4d57-9a2e-d45fbc624969 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:58.060534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89b0e1-9265-4d57-9a2e-d45fbc624969 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.060534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89b0e1-9265-4d57-9a2e-d45fbc624969 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.080947 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1318/2636] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:58.082005 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89b0e1-9265-4d57-9a2e-d45fbc624969 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.084539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcd1b5-9acb-4f3e-96f3-f4766fdd8271 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.093261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d24248-606b-40fe-8b06-4362ea17d3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.093888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d24248-606b-40fe-8b06-4362ea17d3c6 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.094169 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d24248-606b-40fe-8b06-4362ea17d3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.094442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d24248-606b-40fe-8b06-4362ea17d3c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.095087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d24248-606b-40fe-8b06-4362ea17d3c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.097615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d24248-606b-40fe-8b06-4362ea17d3c6 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.099516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89b0e1-9265-4d57-9a2e-d45fbc624969 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.106623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31df5c0-cdef-453b-bd42-1a444d4de72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.107288 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31df5c0-cdef-453b-bd42-1a444d4de72f admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.107602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31df5c0-cdef-453b-bd42-1a444d4de72f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.107876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31df5c0-cdef-453b-bd42-1a444d4de72f admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.108481 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31df5c0-cdef-453b-bd42-1a444d4de72f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.110486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31df5c0-cdef-453b-bd42-1a444d4de72f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.143814 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1319/2637] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:58 2022] POST /identity/v3/projects => generated 407 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:58.147956 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1319/2638] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:58 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:58.149170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d24248-606b-40fe-8b06-4362ea17d3c6 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.155234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31df5c0-cdef-453b-bd42-1a444d4de72f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.238674 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31df5c0-cdef-453b-bd42-1a444d4de72f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.244228 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ec22d-b6e2-41ed-9245-4a657b114d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.244876 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ec22d-b6e2-41ed-9245-4a657b114d5d admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.246316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ec22d-b6e2-41ed-9245-4a657b114d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.246316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ec22d-b6e2-41ed-9245-4a657b114d5d admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.246538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ec22d-b6e2-41ed-9245-4a657b114d5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.248646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ec22d-b6e2-41ed-9245-4a657b114d5d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.259642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d24248-606b-40fe-8b06-4362ea17d3c6 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.273519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972148b3-7ecf-4417-ad7e-aee5bd12abb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.273901 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972148b3-7ecf-4417-ad7e-aee5bd12abb5 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.274221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972148b3-7ecf-4417-ad7e-aee5bd12abb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.274448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972148b3-7ecf-4417-ad7e-aee5bd12abb5 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.275608 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972148b3-7ecf-4417-ad7e-aee5bd12abb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.277660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972148b3-7ecf-4417-ad7e-aee5bd12abb5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.283791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1320/2639] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:58 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:58.285588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ec22d-b6e2-41ed-9245-4a657b114d5d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.298404 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ec22d-b6e2-41ed-9245-4a657b114d5d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.304047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ac52a-9522-4863-8660-669eb5506d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.304835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ac52a-9522-4863-8660-669eb5506d8c admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.305121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ac52a-9522-4863-8660-669eb5506d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.305882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ac52a-9522-4863-8660-669eb5506d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.306030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1320/2640] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:58 2022] POST /identity/v3/users => generated 310 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:58.306430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ac52a-9522-4863-8660-669eb5506d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.308172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ac52a-9522-4863-8660-669eb5506d8c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.311080 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972148b3-7ecf-4417-ad7e-aee5bd12abb5 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.325719 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1321/2641] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:58.326037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972148b3-7ecf-4417-ad7e-aee5bd12abb5 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.326855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ac52a-9522-4863-8660-669eb5506d8c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.332587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bbda8-c5a0-4c42-901c-c00f6f7a2fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.333209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bbda8-c5a0-4c42-901c-c00f6f7a2fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.333489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bbda8-c5a0-4c42-901c-c00f6f7a2fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.334300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bbda8-c5a0-4c42-901c-c00f6f7a2fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.335152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6bbda8-c5a0-4c42-901c-c00f6f7a2fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.337178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6bbda8-c5a0-4c42-901c-c00f6f7a2fb2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.338473 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ac52a-9522-4863-8660-669eb5506d8c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.343976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fddb0-eb34-4f6b-8492-771c8c47f23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.344135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:58.345066 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fddb0-eb34-4f6b-8492-771c8c47f23e admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.345334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fddb0-eb34-4f6b-8492-771c8c47f23e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.345612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fddb0-eb34-4f6b-8492-771c8c47f23e admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.346273 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fddb0-eb34-4f6b-8492-771c8c47f23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.348023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fddb0-eb34-4f6b-8492-771c8c47f23e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.350426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1321/2642] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:55:58.354537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bbda8-c5a0-4c42-901c-c00f6f7a2fb2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.360097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1322/2643] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:55:58.364148 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fddb0-eb34-4f6b-8492-771c8c47f23e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.374003 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bbda8-c5a0-4c42-901c-c00f6f7a2fb2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.375084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fddb0-eb34-4f6b-8492-771c8c47f23e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.378824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8ba48-d7cf-47e9-96f8-a8bcf1dece14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.379533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ba48-d7cf-47e9-96f8-a8bcf1dece14 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.379817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ba48-d7cf-47e9-96f8-a8bcf1dece14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.380918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ba48-d7cf-47e9-96f8-a8bcf1dece14 admin admin] PATH_INFO: `/v3/projects/4890755660834f7fb7c3981d280dc15d/users/b711d3d077784df2b59ed5afa86330e4/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.381038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11e341-788b-44e3-935a-174fbfdbff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.381944 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11e341-788b-44e3-935a-174fbfdbff79 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.382323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11e341-788b-44e3-935a-174fbfdbff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.382666 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11e341-788b-44e3-935a-174fbfdbff79 admin admin] PATH_INFO: `/v3/projects/24bc627cb20b4a07be13995378e10916/users/c60d2d40e00e4733ba47107e1492d56b/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.398210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8ba48-d7cf-47e9-96f8-a8bcf1dece14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4890755660834f7fb7c3981d280dc15d, user_id=b711d3d077784df2b59ed5afa86330e4, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.400484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11e341-788b-44e3-935a-174fbfdbff79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24bc627cb20b4a07be13995378e10916, user_id=c60d2d40e00e4733ba47107e1492d56b, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.401567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8ba48-d7cf-47e9-96f8-a8bcf1dece14 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.403375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11e341-788b-44e3-935a-174fbfdbff79 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.426064 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1322/2644] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:58 2022] PUT /identity/v3/projects/4890755660834f7fb7c3981d280dc15d/users/b711d3d077784df2b59ed5afa86330e4/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:58.431593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8ba48-d7cf-47e9-96f8-a8bcf1dece14 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.439267 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1323/2645] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:58 2022] PUT /identity/v3/projects/24bc627cb20b4a07be13995378e10916/users/c60d2d40e00e4733ba47107e1492d56b/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:58.446056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f9d32-c65b-4d1c-8570-589256958ec7 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.446375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f9d32-c65b-4d1c-8570-589256958ec7 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.446604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f9d32-c65b-4d1c-8570-589256958ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.496304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-636f9d32-c65b-4d1c-8570-589256958ec7 None None] MFA Rules not processed for user `c60d2d40e00e4733ba47107e1492d56b`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:58.497559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-636f9d32-c65b-4d1c-8570-589256958ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:58.530990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8ba48-d7cf-47e9-96f8-a8bcf1dece14 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.544759 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:58.597019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1324/2646] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:58.611782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f9d32-c65b-4d1c-8570-589256958ec7 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.613243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-636f9d32-c65b-4d1c-8570-589256958ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:58.618354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84765f-f515-413a-a64f-d1251179dd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.619402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84765f-f515-413a-a64f-d1251179dd36 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.619860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84765f-f515-413a-a64f-d1251179dd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.620559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84765f-f515-413a-a64f-d1251179dd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.621397 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84765f-f515-413a-a64f-d1251179dd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.623239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84765f-f515-413a-a64f-d1251179dd36 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.640352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1323/2647] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:55:58.644999 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84765f-f515-413a-a64f-d1251179dd36 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.658393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84765f-f515-413a-a64f-d1251179dd36 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.663879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109f9cb-6175-4032-b7dc-e5e7fd3fbf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.664732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109f9cb-6175-4032-b7dc-e5e7fd3fbf9c admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.665009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109f9cb-6175-4032-b7dc-e5e7fd3fbf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.665301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109f9cb-6175-4032-b7dc-e5e7fd3fbf9c admin admin] PATH_INFO: `/v3/projects/4890755660834f7fb7c3981d280dc15d/users/b711d3d077784df2b59ed5afa86330e4/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.667668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109f9cb-6175-4032-b7dc-e5e7fd3fbf9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4890755660834f7fb7c3981d280dc15d, user_id=b711d3d077784df2b59ed5afa86330e4, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.670031 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109f9cb-6175-4032-b7dc-e5e7fd3fbf9c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.685215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f9d32-c65b-4d1c-8570-589256958ec7 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.685967 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1324/2648] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:58 2022] PUT /identity/v3/projects/4890755660834f7fb7c3981d280dc15d/users/b711d3d077784df2b59ed5afa86330e4/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:55:58.697616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd51e4-51b6-4603-9306-7beeaf39765a None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.697913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd51e4-51b6-4603-9306-7beeaf39765a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.698193 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd51e4-51b6-4603-9306-7beeaf39765a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.730210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-81fd51e4-51b6-4603-9306-7beeaf39765a None None] MFA Rules not processed for user `b711d3d077784df2b59ed5afa86330e4`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:58.731496 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-81fd51e4-51b6-4603-9306-7beeaf39765a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:58.744736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa0287-5701-40ba-9b5f-521d5f0869ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.745882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa0287-5701-40ba-9b5f-521d5f0869ce service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.746312 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa0287-5701-40ba-9b5f-521d5f0869ce service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.746586 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa0287-5701-40ba-9b5f-521d5f0869ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.793074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:58.820121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa0287-5701-40ba-9b5f-521d5f0869ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.822552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1325/2649] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:55:58 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:58.823200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa0287-5701-40ba-9b5f-521d5f0869ce service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.839250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1325/2650] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:55:58.839576 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd51e4-51b6-4603-9306-7beeaf39765a None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:58.841216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-81fd51e4-51b6-4603-9306-7beeaf39765a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:58.855210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd51e4-51b6-4603-9306-7beeaf39765a None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:58.859100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a9b9d-ec4b-44a6-b3d1-2109789f67bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:58.859732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a9b9d-ec4b-44a6-b3d1-2109789f67bb service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:58.860013 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a9b9d-ec4b-44a6-b3d1-2109789f67bb service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:58.860291 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a9b9d-ec4b-44a6-b3d1-2109789f67bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:58.871526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a9b9d-ec4b-44a6-b3d1-2109789f67bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:58.878579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a9b9d-ec4b-44a6-b3d1-2109789f67bb service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:58.890309 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1326/2651] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:58 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:55:59.087300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa0287-5701-40ba-9b5f-521d5f0869ce service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:59.102843 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa0287-5701-40ba-9b5f-521d5f0869ce service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:59.109084 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0265a-7311-4858-9a61-ac16a9cfa5b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:59.110161 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0265a-7311-4858-9a61-ac16a9cfa5b0 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:59.110672 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0265a-7311-4858-9a61-ac16a9cfa5b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:59.111186 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0265a-7311-4858-9a61-ac16a9cfa5b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:59.167586 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:59.188449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0265a-7311-4858-9a61-ac16a9cfa5b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:59.189960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0265a-7311-4858-9a61-ac16a9cfa5b0 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:59.201227 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1326/2652] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:59 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:56:01.578352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a9b9d-ec4b-44a6-b3d1-2109789f67bb service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:01.644301 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a9b9d-ec4b-44a6-b3d1-2109789f67bb service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:01.650140 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:01.650775 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:01.651054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:01.651354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] PATH_INFO: `/v3/users/c60d2d40e00e4733ba47107e1492d56b` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:01.652503 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c60d2d40e00e4733ba47107e1492d56b)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:01.654440 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:01.703687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c60d2d40e00e4733ba47107e1492d56b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:01.717603 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c60d2d40e00e4733ba47107e1492d56b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:01.718373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c60d2d40e00e4733ba47107e1492d56b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:01.719000 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c60d2d40e00e4733ba47107e1492d56b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:01.728228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c60d2d40e00e4733ba47107e1492d56b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:01.745868 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1327/2653] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:56:01 2022] DELETE /identity/v3/users/c60d2d40e00e4733ba47107e1492d56b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:01.752702 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61dcfb-4d16-4247-aded-a1d31bcf05a6 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:01.753701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61dcfb-4d16-4247-aded-a1d31bcf05a6 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:01.753701 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61dcfb-4d16-4247-aded-a1d31bcf05a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:01.777376 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-cf61dcfb-4d16-4247-aded-a1d31bcf05a6 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:01.778752 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-cf61dcfb-4d16-4247-aded-a1d31bcf05a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:01.841898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1327/2654] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:01.852124 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:01.853721 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:01.918408 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b83a8-276d-4787-91bf-f52b963efdb5 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:01.922196 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df11045-6d9d-4ba7-a757-42897631a4e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:01.922807 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df11045-6d9d-4ba7-a757-42897631a4e3 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:01.923087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df11045-6d9d-4ba7-a757-42897631a4e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:01.923388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df11045-6d9d-4ba7-a757-42897631a4e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:01.934076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df11045-6d9d-4ba7-a757-42897631a4e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:01.935865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df11045-6d9d-4ba7-a757-42897631a4e3 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:01.941930 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:01.948160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1328/2655] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:02.256398 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61dcfb-4d16-4247-aded-a1d31bcf05a6 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:02.257875 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-cf61dcfb-4d16-4247-aded-a1d31bcf05a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:02.272546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61dcfb-4d16-4247-aded-a1d31bcf05a6 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:02.281187 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:02.281842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:02.282117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:02.282383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36 admin admin] PATH_INFO: `/v3/projects/24bc627cb20b4a07be13995378e10916` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:02.283640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24bc627cb20b4a07be13995378e10916)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:02.291920 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:02.347904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24bc627cb20b4a07be13995378e10916', 'request_id': 'req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:02.354954 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24bc627cb20b4a07be13995378e10916', 'request_id': 'req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:02.360479 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24bc627cb20b4a07be13995378e10916', 'request_id': 'req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:02.373984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36 admin admin] The token cache is being invalidate because project 24bc627cb20b4a07be13995378e10916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:56:02.374410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:02.374950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-84ea3c6d-655d-4a4b-bcab-70b0f771ee36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:02.376577 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1328/2656] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:56:02 2022] DELETE /identity/v3/projects/24bc627cb20b4a07be13995378e10916 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:02.381646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85ab64-920d-4c7b-87b2-97ad3034e0ea None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:02.381931 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85ab64-920d-4c7b-87b2-97ad3034e0ea None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:02.382199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85ab64-920d-4c7b-87b2-97ad3034e0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:02.403248 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-2d85ab64-920d-4c7b-87b2-97ad3034e0ea None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:02.404471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2d85ab64-920d-4c7b-87b2-97ad3034e0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:02.471087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1329/2657] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:02.479370 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9b540-21c4-447d-8f5e-f473c2a5647e None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:02.479656 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9b540-21c4-447d-8f5e-f473c2a5647e None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:02.479932 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9b540-21c4-447d-8f5e-f473c2a5647e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:02.500915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-39b9b540-21c4-447d-8f5e-f473c2a5647e None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:02.502119 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-39b9b540-21c4-447d-8f5e-f473c2a5647e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:02.513081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1329/2658] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:02.517915 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85ab64-920d-4c7b-87b2-97ad3034e0ea None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:02.528452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85ab64-920d-4c7b-87b2-97ad3034e0ea None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:02.533482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa2a8a-36f4-4664-9e07-c8e741573f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:02.534091 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa2a8a-36f4-4664-9e07-c8e741573f67 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:02.534367 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa2a8a-36f4-4664-9e07-c8e741573f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:02.534657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa2a8a-36f4-4664-9e07-c8e741573f67 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:02.535356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa2a8a-36f4-4664-9e07-c8e741573f67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:56:02.535653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa2a8a-36f4-4664-9e07-c8e741573f67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:02.537842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa2a8a-36f4-4664-9e07-c8e741573f67 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:02.549976 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1330/2659] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:56:02.555883 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9b540-21c4-447d-8f5e-f473c2a5647e None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:02.565012 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9b540-21c4-447d-8f5e-f473c2a5647e None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:02.569964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031c296-4af3-41cc-9a08-2de1385ed3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:02.570572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031c296-4af3-41cc-9a08-2de1385ed3b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:02.570849 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031c296-4af3-41cc-9a08-2de1385ed3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:02.571158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031c296-4af3-41cc-9a08-2de1385ed3b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:02.571797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031c296-4af3-41cc-9a08-2de1385ed3b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:02.573705 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031c296-4af3-41cc-9a08-2de1385ed3b0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:02.603445 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1330/2660] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:56:02 2022] POST /identity/v3/projects => generated 397 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:02.607875 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa2a8a-36f4-4664-9e07-c8e741573f67 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:02.619572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa2a8a-36f4-4664-9e07-c8e741573f67 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:02.682744 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7c75-31b6-4aa8-bd8b-de611a899fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:02.683430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7c75-31b6-4aa8-bd8b-de611a899fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:02.683713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7c75-31b6-4aa8-bd8b-de611a899fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:02.683990 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7c75-31b6-4aa8-bd8b-de611a899fd4 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:02.684639 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d7c75-31b6-4aa8-bd8b-de611a899fd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:02.687006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d7c75-31b6-4aa8-bd8b-de611a899fd4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:02.709620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1331/2661] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:56:02 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:02.714329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031c296-4af3-41cc-9a08-2de1385ed3b0 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:02.726176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031c296-4af3-41cc-9a08-2de1385ed3b0 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:02.731016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ff01e-21c3-4e89-bd0e-0a88670c65d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:02.731670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ff01e-21c3-4e89-bd0e-0a88670c65d4 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:02.731946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ff01e-21c3-4e89-bd0e-0a88670c65d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:02.732216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ff01e-21c3-4e89-bd0e-0a88670c65d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:02.732868 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ff01e-21c3-4e89-bd0e-0a88670c65d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:02.734485 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ff01e-21c3-4e89-bd0e-0a88670c65d4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:02.747053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1331/2662] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:02.751582 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7c75-31b6-4aa8-bd8b-de611a899fd4 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:02.762570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7c75-31b6-4aa8-bd8b-de611a899fd4 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:02.767375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a34c13-ff35-4339-81fe-ff8f306e785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:02.767987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a34c13-ff35-4339-81fe-ff8f306e785d admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:02.768284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a34c13-ff35-4339-81fe-ff8f306e785d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:02.768543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a34c13-ff35-4339-81fe-ff8f306e785d admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:02.769196 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a34c13-ff35-4339-81fe-ff8f306e785d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:02.770835 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a34c13-ff35-4339-81fe-ff8f306e785d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:02.783329 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1332/2663] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:02.788910 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ff01e-21c3-4e89-bd0e-0a88670c65d4 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:02.799609 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ff01e-21c3-4e89-bd0e-0a88670c65d4 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:02.805276 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6b871-171c-426a-9a04-0124211d8143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:02.805940 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6b871-171c-426a-9a04-0124211d8143 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:02.806218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6b871-171c-426a-9a04-0124211d8143 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:02.806486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6b871-171c-426a-9a04-0124211d8143 admin admin] PATH_INFO: `/v3/projects/4bd92510259e49918e147a24a1eefc29/users/4cf46c3a9a2a4a2e8c8434c771f4bd37/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:02.824167 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6b871-171c-426a-9a04-0124211d8143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd92510259e49918e147a24a1eefc29, user_id=4cf46c3a9a2a4a2e8c8434c771f4bd37, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:02.826825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6b871-171c-426a-9a04-0124211d8143 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:02.841622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1332/2664] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:56:02 2022] PUT /identity/v3/projects/4bd92510259e49918e147a24a1eefc29/users/4cf46c3a9a2a4a2e8c8434c771f4bd37/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:56:02.853049 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26e75a-094c-4559-8ffe-d69e0ec2de57 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:02.853337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26e75a-094c-4559-8ffe-d69e0ec2de57 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:02.853623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26e75a-094c-4559-8ffe-d69e0ec2de57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:02.884809 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ac26e75a-094c-4559-8ffe-d69e0ec2de57 None None] MFA Rules not processed for user `4cf46c3a9a2a4a2e8c8434c771f4bd37`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:02.886034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ac26e75a-094c-4559-8ffe-d69e0ec2de57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:02.955745 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:02.968998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1333/2665] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:02 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:56:02.982784 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6b871-171c-426a-9a04-0124211d8143 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:02.985081 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2df6b871-171c-426a-9a04-0124211d8143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:02.995941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:03.071865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6b871-171c-426a-9a04-0124211d8143 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:03.075893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc3d22-b77f-46f5-becd-1b89eb2098e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:03.076561 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc3d22-b77f-46f5-becd-1b89eb2098e9 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:03.076825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc3d22-b77f-46f5-becd-1b89eb2098e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:03.077099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc3d22-b77f-46f5-becd-1b89eb2098e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:03.088330 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc3d22-b77f-46f5-becd-1b89eb2098e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:03.090675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc3d22-b77f-46f5-becd-1b89eb2098e9 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:03.102772 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1333/2666] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:02 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:09.116489 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26e75a-094c-4559-8ffe-d69e0ec2de57 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:09.124562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ac26e75a-094c-4559-8ffe-d69e0ec2de57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:09.219255 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26e75a-094c-4559-8ffe-d69e0ec2de57 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:09.224693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:09.225964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:09.226279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:09.226559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] PATH_INFO: `/v3/users/4dd59dfda97d40aaaa9706ab1029a352` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:09.227870 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dd59dfda97d40aaaa9706ab1029a352)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:09.229846 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:09.312276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dd59dfda97d40aaaa9706ab1029a352'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:09.327738 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dd59dfda97d40aaaa9706ab1029a352'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:09.328475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dd59dfda97d40aaaa9706ab1029a352'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:09.329167 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dd59dfda97d40aaaa9706ab1029a352'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:09.342974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dd59dfda97d40aaaa9706ab1029a352'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:09.359694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1334/2667] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:56:09 2022] DELETE /identity/v3/users/4dd59dfda97d40aaaa9706ab1029a352 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:09.364579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f9f15-1602-45cd-aea5-a71981846803 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:09.365587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f9f15-1602-45cd-aea5-a71981846803 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:09.365961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f9f15-1602-45cd-aea5-a71981846803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:09.390162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-547f9f15-1602-45cd-aea5-a71981846803 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:09.391625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-547f9f15-1602-45cd-aea5-a71981846803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:09.462265 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1334/2668] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:09.472762 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:09.474531 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:09.538836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d7d72-f323-4550-b9b3-8266dc625fab admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:09.542788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fd527-b802-4a4a-b33a-1b5c41f6360f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:09.543450 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fd527-b802-4a4a-b33a-1b5c41f6360f service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:09.543735 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fd527-b802-4a4a-b33a-1b5c41f6360f service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:09.544008 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fd527-b802-4a4a-b33a-1b5c41f6360f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:09.555758 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fd527-b802-4a4a-b33a-1b5c41f6360f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:09.557250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fd527-b802-4a4a-b33a-1b5c41f6360f service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:09.568843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1335/2669] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:09.916539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f9f15-1602-45cd-aea5-a71981846803 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:09.918582 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-547f9f15-1602-45cd-aea5-a71981846803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:09.936545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f9f15-1602-45cd-aea5-a71981846803 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:09.945026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:09.946224 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:09.946898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:09.947232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] PATH_INFO: `/v3/projects/e39f9909594c4a0d9e148ed3b443bde9` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:09.949552 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e39f9909594c4a0d9e148ed3b443bde9)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:09.953315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:10.019957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e39f9909594c4a0d9e148ed3b443bde9', 'request_id': 'req-cb62271f-62ce-409d-84e8-1887978f904e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.027315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e39f9909594c4a0d9e148ed3b443bde9', 'request_id': 'req-cb62271f-62ce-409d-84e8-1887978f904e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.033763 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e39f9909594c4a0d9e148ed3b443bde9', 'request_id': 'req-cb62271f-62ce-409d-84e8-1887978f904e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.049198 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] The token cache is being invalidate because project e39f9909594c4a0d9e148ed3b443bde9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:56:10.049523 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.050182 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.052638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1335/2670] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:56:09 2022] DELETE /identity/v3/projects/e39f9909594c4a0d9e148ed3b443bde9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:56:10.056743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fd527-b802-4a4a-b33a-1b5c41f6360f service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:10.067145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fd527-b802-4a4a-b33a-1b5c41f6360f service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:10.085029 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:10.140449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:10.141431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:10.141863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:10.143502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] PATH_INFO: `/v3/users/b711d3d077784df2b59ed5afa86330e4` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:10.144806 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b711d3d077784df2b59ed5afa86330e4)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:10.147004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:10.198785 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b711d3d077784df2b59ed5afa86330e4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.211695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b711d3d077784df2b59ed5afa86330e4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.212441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b711d3d077784df2b59ed5afa86330e4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.213147 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b711d3d077784df2b59ed5afa86330e4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.222283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b711d3d077784df2b59ed5afa86330e4'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.237270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1336/2671] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:56:10 2022] DELETE /identity/v3/users/b711d3d077784df2b59ed5afa86330e4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:56:10.431431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:10.433828 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:10.452976 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62271f-62ce-409d-84e8-1887978f904e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:10.502271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:10.566919 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946843b-5c44-4cea-94f3-f7b90b949fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:10.568256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946843b-5c44-4cea-94f3-f7b90b949fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:10.568542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946843b-5c44-4cea-94f3-f7b90b949fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:10.568820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946843b-5c44-4cea-94f3-f7b90b949fb6 admin admin] PATH_INFO: `/v3/users/4cf46c3a9a2a4a2e8c8434c771f4bd37` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:10.570524 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946843b-5c44-4cea-94f3-f7b90b949fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf46c3a9a2a4a2e8c8434c771f4bd37)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:10.572731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946843b-5c44-4cea-94f3-f7b90b949fb6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:10.630955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:10.633761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:10.651034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e5568-217a-45b5-a2bf-4ae3f60fe57b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:10.656782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:10.657434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:10.657850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:10.658012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a admin admin] PATH_INFO: `/v3/projects/4890755660834f7fb7c3981d280dc15d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:10.659325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4890755660834f7fb7c3981d280dc15d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:10.662461 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:10.690451 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5946843b-5c44-4cea-94f3-f7b90b949fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf46c3a9a2a4a2e8c8434c771f4bd37'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.707019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5946843b-5c44-4cea-94f3-f7b90b949fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf46c3a9a2a4a2e8c8434c771f4bd37'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.709441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5946843b-5c44-4cea-94f3-f7b90b949fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf46c3a9a2a4a2e8c8434c771f4bd37'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.737034 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5946843b-5c44-4cea-94f3-f7b90b949fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf46c3a9a2a4a2e8c8434c771f4bd37'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.737997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5946843b-5c44-4cea-94f3-f7b90b949fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf46c3a9a2a4a2e8c8434c771f4bd37'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.760492 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4890755660834f7fb7c3981d280dc15d', 'request_id': 'req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.761598 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1336/2672] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:56:10 2022] DELETE /identity/v3/users/4cf46c3a9a2a4a2e8c8434c771f4bd37 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:10.766032 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36906c0e-604e-413c-b675-dd864191fe59 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:10.766282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36906c0e-604e-413c-b675-dd864191fe59 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:10.766903 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36906c0e-604e-413c-b675-dd864191fe59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:10.774267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4890755660834f7fb7c3981d280dc15d', 'request_id': 'req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.786138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4890755660834f7fb7c3981d280dc15d', 'request_id': 'req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.791635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-36906c0e-604e-413c-b675-dd864191fe59 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:10.793375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-36906c0e-604e-413c-b675-dd864191fe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:10.806090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a admin admin] The token cache is being invalidate because project 4890755660834f7fb7c3981d280dc15d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:56:10.807283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.808017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-709c9fea-7ec3-43c7-bb1d-eaf4b892444a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:10.809747 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1337/2673] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:56:10 2022] DELETE /identity/v3/projects/4890755660834f7fb7c3981d280dc15d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:56:10.816142 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ef206-bca1-41e6-8ee4-ca4db820de43 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:10.816793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ef206-bca1-41e6-8ee4-ca4db820de43 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:10.817137 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ef206-bca1-41e6-8ee4-ca4db820de43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:10.842707 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-598ef206-bca1-41e6-8ee4-ca4db820de43 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:10.844034 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-598ef206-bca1-41e6-8ee4-ca4db820de43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:10.862642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:10.896405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1337/2674] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:56:10.910634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36906c0e-604e-413c-b675-dd864191fe59 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:10.913928 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-36906c0e-604e-413c-b675-dd864191fe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:11.020893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:11.030243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36906c0e-604e-413c-b675-dd864191fe59 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:11.046314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186a889-e4fc-4488-9110-7ea08ed32235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:11.047008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186a889-e4fc-4488-9110-7ea08ed32235 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.047659 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186a889-e4fc-4488-9110-7ea08ed32235 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.047946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186a889-e4fc-4488-9110-7ea08ed32235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.061513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186a889-e4fc-4488-9110-7ea08ed32235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:11.063612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186a889-e4fc-4488-9110-7ea08ed32235 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:11.080413 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1338/2675] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:11.103535 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1338/2676] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:56:11.112768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdd143-64a6-4eb5-8e1e-ac86dc5ef279 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.113057 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdd143-64a6-4eb5-8e1e-ac86dc5ef279 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.113329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdd143-64a6-4eb5-8e1e-ac86dc5ef279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.137021 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-61cdd143-64a6-4eb5-8e1e-ac86dc5ef279 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:11.138286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-61cdd143-64a6-4eb5-8e1e-ac86dc5ef279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:11.151387 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1339/2677] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:11.155956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ef206-bca1-41e6-8ee4-ca4db820de43 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:11.168227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ef206-bca1-41e6-8ee4-ca4db820de43 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:11.174502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7092721-1c83-457b-9d04-30f0b099c09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:11.175568 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7092721-1c83-457b-9d04-30f0b099c09e admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.175731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7092721-1c83-457b-9d04-30f0b099c09e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.176036 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7092721-1c83-457b-9d04-30f0b099c09e admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.176879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7092721-1c83-457b-9d04-30f0b099c09e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:56:11.177128 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7092721-1c83-457b-9d04-30f0b099c09e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:11.179354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7092721-1c83-457b-9d04-30f0b099c09e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:11.199623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1339/2678] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:56:11.200662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdd143-64a6-4eb5-8e1e-ac86dc5ef279 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:11.214054 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdd143-64a6-4eb5-8e1e-ac86dc5ef279 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:11.221118 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6202f1a-7e87-4eff-bd42-024055a459a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:11.221834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6202f1a-7e87-4eff-bd42-024055a459a2 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.222126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6202f1a-7e87-4eff-bd42-024055a459a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.222395 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6202f1a-7e87-4eff-bd42-024055a459a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.223098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6202f1a-7e87-4eff-bd42-024055a459a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:11.225718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6202f1a-7e87-4eff-bd42-024055a459a2 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:11.257585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1340/2679] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:56:11 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:11.261792 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7092721-1c83-457b-9d04-30f0b099c09e admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:11.273406 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7092721-1c83-457b-9d04-30f0b099c09e admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:11.350882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ae6a6-070a-454a-9681-43b2040a3d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:11.352016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ae6a6-070a-454a-9681-43b2040a3d2f admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.352393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ae6a6-070a-454a-9681-43b2040a3d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.352763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ae6a6-070a-454a-9681-43b2040a3d2f admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.353482 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ae6a6-070a-454a-9681-43b2040a3d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:11.355646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ae6a6-070a-454a-9681-43b2040a3d2f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:11.386004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1340/2680] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:56:11 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:11.392715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6202f1a-7e87-4eff-bd42-024055a459a2 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:11.410256 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6202f1a-7e87-4eff-bd42-024055a459a2 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:11.416888 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da50ca-c4da-483c-bc4b-f3de95ec7c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:11.417517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da50ca-c4da-483c-bc4b-f3de95ec7c1d admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.417788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da50ca-c4da-483c-bc4b-f3de95ec7c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.418162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da50ca-c4da-483c-bc4b-f3de95ec7c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.419010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da50ca-c4da-483c-bc4b-f3de95ec7c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:11.420872 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da50ca-c4da-483c-bc4b-f3de95ec7c1d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:11.442311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1341/2681] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:11.448496 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ae6a6-070a-454a-9681-43b2040a3d2f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:11.465533 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ae6a6-070a-454a-9681-43b2040a3d2f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:11.481591 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24226b64-4edb-4151-a9cb-7090c12e6273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:11.482781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24226b64-4edb-4151-a9cb-7090c12e6273 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.483395 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24226b64-4edb-4151-a9cb-7090c12e6273 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.483957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24226b64-4edb-4151-a9cb-7090c12e6273 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.485263 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24226b64-4edb-4151-a9cb-7090c12e6273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:11.488575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24226b64-4edb-4151-a9cb-7090c12e6273 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:11.510527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1341/2682] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:56:11.515513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da50ca-c4da-483c-bc4b-f3de95ec7c1d admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:11.525728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24226b64-4edb-4151-a9cb-7090c12e6273 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:11.528209 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-24226b64-4edb-4151-a9cb-7090c12e6273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:11.533861 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da50ca-c4da-483c-bc4b-f3de95ec7c1d admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:11.544633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f18ad-7a7c-47db-9389-fc28cbdf824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:11.545764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f18ad-7a7c-47db-9389-fc28cbdf824c admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.546260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f18ad-7a7c-47db-9389-fc28cbdf824c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.546971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f18ad-7a7c-47db-9389-fc28cbdf824c admin admin] PATH_INFO: `/v3/projects/1604e65b8c154dd491b8b2ab1747e80a/users/06c2c2ac5afc4a85946b1bc6a959762a/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.555953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24226b64-4edb-4151-a9cb-7090c12e6273 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:11.562202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:11.562852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.563146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.563445 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] PATH_INFO: `/v3/projects/4bd92510259e49918e147a24a1eefc29` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.564842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd92510259e49918e147a24a1eefc29)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:11.567089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:11.579343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f18ad-7a7c-47db-9389-fc28cbdf824c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1604e65b8c154dd491b8b2ab1747e80a, user_id=06c2c2ac5afc4a85946b1bc6a959762a, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:11.586502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f18ad-7a7c-47db-9389-fc28cbdf824c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:11.609409 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1342/2683] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:56:11 2022] PUT /identity/v3/projects/1604e65b8c154dd491b8b2ab1747e80a/users/06c2c2ac5afc4a85946b1bc6a959762a/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:11.617460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce241425-01dc-4a7b-95fd-571350df7afc None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.618127 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce241425-01dc-4a7b-95fd-571350df7afc None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.618625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce241425-01dc-4a7b-95fd-571350df7afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.634100 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd92510259e49918e147a24a1eefc29', 'request_id': 'req-e65f94cd-fc22-4259-9409-899feb60fcd2'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:11.645283 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd92510259e49918e147a24a1eefc29', 'request_id': 'req-e65f94cd-fc22-4259-9409-899feb60fcd2'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:11.649587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-ce241425-01dc-4a7b-95fd-571350df7afc None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:11.651173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ce241425-01dc-4a7b-95fd-571350df7afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:11.662059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd92510259e49918e147a24a1eefc29', 'request_id': 'req-e65f94cd-fc22-4259-9409-899feb60fcd2'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:11.684571 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] The token cache is being invalidate because project 4bd92510259e49918e147a24a1eefc29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:56:11.684909 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:11.685885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:11.688635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1342/2684] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:56:11 2022] DELETE /identity/v3/projects/4bd92510259e49918e147a24a1eefc29 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:56:11.783888 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1343/2685] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:11.802728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:11.805876 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:11.826543 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:11.901369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f94cd-fc22-4259-9409-899feb60fcd2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:11.906222 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6b8af-fe90-4b86-ac80-ec3385abc549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:11.906996 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6b8af-fe90-4b86-ac80-ec3385abc549 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.907508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6b8af-fe90-4b86-ac80-ec3385abc549 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.907710 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6b8af-fe90-4b86-ac80-ec3385abc549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.920667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6b8af-fe90-4b86-ac80-ec3385abc549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:11.923537 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6b8af-fe90-4b86-ac80-ec3385abc549 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:11.954148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1343/2686] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:12.761442 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbee3cf-25af-4651-af0d-4826a437e9fd None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:12.762004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbee3cf-25af-4651-af0d-4826a437e9fd None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:12.762004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbee3cf-25af-4651-af0d-4826a437e9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:12.799053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-4cbee3cf-25af-4651-af0d-4826a437e9fd None None] MFA Rules not processed for user `06c2c2ac5afc4a85946b1bc6a959762a`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:12.800327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-4cbee3cf-25af-4651-af0d-4826a437e9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:12.892518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1344/2687] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:56:12.899237 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6b8af-fe90-4b86-ac80-ec3385abc549 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:12.901066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-21e6b8af-fe90-4b86-ac80-ec3385abc549 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:12.916393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6b8af-fe90-4b86-ac80-ec3385abc549 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:12.925300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede6843-f52b-4109-8048-c4526407ea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:12.925300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede6843-f52b-4109-8048-c4526407ea97 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:12.925300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede6843-f52b-4109-8048-c4526407ea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:12.925300 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede6843-f52b-4109-8048-c4526407ea97 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:12.925768 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede6843-f52b-4109-8048-c4526407ea97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:12.925768 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede6843-f52b-4109-8048-c4526407ea97 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:12.956179 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1344/2688] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:56:12 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:56:12.964336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbee3cf-25af-4651-af0d-4826a437e9fd None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:12.977483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbee3cf-25af-4651-af0d-4826a437e9fd None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:13.006461 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:13.080140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c7ee-ab15-4a94-87cc-f107d785a951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:13.080771 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c7ee-ab15-4a94-87cc-f107d785a951 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:13.081038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c7ee-ab15-4a94-87cc-f107d785a951 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:13.081304 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c7ee-ab15-4a94-87cc-f107d785a951 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:13.081937 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5c7ee-ab15-4a94-87cc-f107d785a951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:13.084699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5c7ee-ab15-4a94-87cc-f107d785a951 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:13.112891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1345/2689] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:56:12 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:13.118508 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede6843-f52b-4109-8048-c4526407ea97 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:13.130279 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede6843-f52b-4109-8048-c4526407ea97 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:13.136681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897236f1-b524-4184-8778-a5a20d4ec597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:13.137256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897236f1-b524-4184-8778-a5a20d4ec597 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:13.137681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897236f1-b524-4184-8778-a5a20d4ec597 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:13.137908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897236f1-b524-4184-8778-a5a20d4ec597 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:13.138470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897236f1-b524-4184-8778-a5a20d4ec597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:13.140218 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897236f1-b524-4184-8778-a5a20d4ec597 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:13.152843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1345/2690] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:56:13.157533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c7ee-ab15-4a94-87cc-f107d785a951 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:13.173714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c7ee-ab15-4a94-87cc-f107d785a951 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:13.180404 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e41e32-fef9-41ce-bbda-60addf292aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:13.181087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e41e32-fef9-41ce-bbda-60addf292aba admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:13.181377 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e41e32-fef9-41ce-bbda-60addf292aba admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:13.181653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e41e32-fef9-41ce-bbda-60addf292aba admin admin] PATH_INFO: `/v3/projects/153cc4b7aaa94a4d99c5dc9f514c29a5/users/3c93a22463ef4a8c918d33a7eea9a9e5/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:13.200023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e41e32-fef9-41ce-bbda-60addf292aba admin admin] RBAC: Authorizing `identity:create_grant(project_id=153cc4b7aaa94a4d99c5dc9f514c29a5, user_id=3c93a22463ef4a8c918d33a7eea9a9e5, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:13.202901 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e41e32-fef9-41ce-bbda-60addf292aba admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:13.217878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1346/2691] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:56:13 2022] PUT /identity/v3/projects/153cc4b7aaa94a4d99c5dc9f514c29a5/users/3c93a22463ef4a8c918d33a7eea9a9e5/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:13.223369 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897236f1-b524-4184-8778-a5a20d4ec597 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:13.235770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897236f1-b524-4184-8778-a5a20d4ec597 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:13.317781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2c7ae-63cc-4299-8c26-7cb7b737a6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:13.318549 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2c7ae-63cc-4299-8c26-7cb7b737a6fd admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:13.318977 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2c7ae-63cc-4299-8c26-7cb7b737a6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:13.319370 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2c7ae-63cc-4299-8c26-7cb7b737a6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:13.320509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2c7ae-63cc-4299-8c26-7cb7b737a6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:13.323705 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2c7ae-63cc-4299-8c26-7cb7b737a6fd admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:13.343908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1346/2692] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:13.349434 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e41e32-fef9-41ce-bbda-60addf292aba admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:13.368156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e41e32-fef9-41ce-bbda-60addf292aba admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:13.376626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d38f77-74ce-4e83-a0fe-e667130cb8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:13.377695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d38f77-74ce-4e83-a0fe-e667130cb8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:13.378193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d38f77-74ce-4e83-a0fe-e667130cb8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:13.378686 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d38f77-74ce-4e83-a0fe-e667130cb8f7 admin admin] PATH_INFO: `/v3/projects/153cc4b7aaa94a4d99c5dc9f514c29a5/users/3c93a22463ef4a8c918d33a7eea9a9e5/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:13.382408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d38f77-74ce-4e83-a0fe-e667130cb8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=153cc4b7aaa94a4d99c5dc9f514c29a5, user_id=3c93a22463ef4a8c918d33a7eea9a9e5, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:13.387882 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d38f77-74ce-4e83-a0fe-e667130cb8f7 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:13.404218 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1347/2693] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:56:13 2022] PUT /identity/v3/projects/153cc4b7aaa94a4d99c5dc9f514c29a5/users/3c93a22463ef4a8c918d33a7eea9a9e5/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:56:14.218597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023814ef-4678-47a1-b24c-1a7b406d33b2 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:14.219226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023814ef-4678-47a1-b24c-1a7b406d33b2 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:14.219226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023814ef-4678-47a1-b24c-1a7b406d33b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:14.252405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-023814ef-4678-47a1-b24c-1a7b406d33b2 None None] MFA Rules not processed for user `3c93a22463ef4a8c918d33a7eea9a9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:14.253670 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-023814ef-4678-47a1-b24c-1a7b406d33b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:14.361152 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:14.368053 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1347/2694] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:14.381551 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d38f77-74ce-4e83-a0fe-e667130cb8f7 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:14.383717 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-02d38f77-74ce-4e83-a0fe-e667130cb8f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:14.462496 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d38f77-74ce-4e83-a0fe-e667130cb8f7 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:14.469217 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc1a66-2f2c-489b-a941-dfd6f995ae3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:14.470283 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc1a66-2f2c-489b-a941-dfd6f995ae3a service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:14.470794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc1a66-2f2c-489b-a941-dfd6f995ae3a service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:14.471337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc1a66-2f2c-489b-a941-dfd6f995ae3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:14.562534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc1a66-2f2c-489b-a941-dfd6f995ae3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:14.564053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc1a66-2f2c-489b-a941-dfd6f995ae3a service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:14.574098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1348/2695] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:16.159757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023814ef-4678-47a1-b24c-1a7b406d33b2 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:16.173857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023814ef-4678-47a1-b24c-1a7b406d33b2 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:16.178691 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4211cf8-66ba-41c0-a98b-3f02c4740b92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:16.179357 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4211cf8-66ba-41c0-a98b-3f02c4740b92 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:16.179676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4211cf8-66ba-41c0-a98b-3f02c4740b92 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:16.179952 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4211cf8-66ba-41c0-a98b-3f02c4740b92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:16.190212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4211cf8-66ba-41c0-a98b-3f02c4740b92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:16.191734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4211cf8-66ba-41c0-a98b-3f02c4740b92 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:16.203239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1348/2696] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:23.184436 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc1a66-2f2c-489b-a941-dfd6f995ae3a service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:23.207459 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc1a66-2f2c-489b-a941-dfd6f995ae3a service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:23.212000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafc0ba-1f3f-4746-a6e6-05075c87e5d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:23.212757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafc0ba-1f3f-4746-a6e6-05075c87e5d9 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:23.213138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafc0ba-1f3f-4746-a6e6-05075c87e5d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:23.213594 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafc0ba-1f3f-4746-a6e6-05075c87e5d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:23.215726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5eafc0ba-1f3f-4746-a6e6-05075c87e5d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:23.227883 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:23.324549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafc0ba-1f3f-4746-a6e6-05075c87e5d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:23.326003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafc0ba-1f3f-4746-a6e6-05075c87e5d9 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:23.337105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1349/2697] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:26.197755 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=6 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:32.603808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eb1de-f8f3-403e-9bad-39d2214d050b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:32.604825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eb1de-f8f3-403e-9bad-39d2214d050b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:32.604825 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eb1de-f8f3-403e-9bad-39d2214d050b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:32.626398 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e13eb1de-f8f3-403e-9bad-39d2214d050b None None] MFA Rules not processed for user `893e06c5ea99497fae2847399b7a1a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:32.627824 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e13eb1de-f8f3-403e-9bad-39d2214d050b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:32.651312 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1349/2698] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:56:33.331638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=32 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:33.566428 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafc0ba-1f3f-4746-a6e6-05075c87e5d9 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:33.706052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafc0ba-1f3f-4746-a6e6-05075c87e5d9 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:33.711364 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:33.712172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:33.712646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:33.712924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] PATH_INFO: `/v3/users/06c2c2ac5afc4a85946b1bc6a959762a` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:33.714270 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c2c2ac5afc4a85946b1bc6a959762a)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:33.716694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:33.779050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c2c2ac5afc4a85946b1bc6a959762a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:33.790972 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c2c2ac5afc4a85946b1bc6a959762a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:33.792097 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c2c2ac5afc4a85946b1bc6a959762a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:33.806707 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c2c2ac5afc4a85946b1bc6a959762a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:33.807536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c2c2ac5afc4a85946b1bc6a959762a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:33.847079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1350/2699] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:56:33 2022] DELETE /identity/v3/users/06c2c2ac5afc4a85946b1bc6a959762a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:34.228436 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13eb1de-f8f3-403e-9bad-39d2214d050b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:34.250790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13eb1de-f8f3-403e-9bad-39d2214d050b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:34.361727 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:34.362575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:34.362942 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:34.363306 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] PATH_INFO: `/v3/projects/1604e65b8c154dd491b8b2ab1747e80a` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:34.364785 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1604e65b8c154dd491b8b2ab1747e80a)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:34.366837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:34.427719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1604e65b8c154dd491b8b2ab1747e80a', 'request_id': 'req-6ea2423e-0b95-4ec0-962a-cfc973856984'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:34.438942 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1604e65b8c154dd491b8b2ab1747e80a', 'request_id': 'req-6ea2423e-0b95-4ec0-962a-cfc973856984'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:34.448010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1604e65b8c154dd491b8b2ab1747e80a', 'request_id': 'req-6ea2423e-0b95-4ec0-962a-cfc973856984'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:34.476130 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] The token cache is being invalidate because project 1604e65b8c154dd491b8b2ab1747e80a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:56:34.476734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:34.477529 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:34.479510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1350/2700] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:56:34 2022] DELETE /identity/v3/projects/1604e65b8c154dd491b8b2ab1747e80a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:34.483480 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:34.485516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:34.501342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb12a66-28b9-49ef-8bec-d2232d3e0580 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:34.573109 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:34.588612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce832f-a458-407b-baff-020e3b6db8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:34.589587 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce832f-a458-407b-baff-020e3b6db8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:34.590085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce832f-a458-407b-baff-020e3b6db8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:34.590375 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce832f-a458-407b-baff-020e3b6db8a0 admin admin] PATH_INFO: `/v3/users/3c93a22463ef4a8c918d33a7eea9a9e5` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:34.599506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce832f-a458-407b-baff-020e3b6db8a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c93a22463ef4a8c918d33a7eea9a9e5)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:34.601602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce832f-a458-407b-baff-020e3b6db8a0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:34.666716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-67ce832f-a458-407b-baff-020e3b6db8a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c93a22463ef4a8c918d33a7eea9a9e5'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:34.677477 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-67ce832f-a458-407b-baff-020e3b6db8a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c93a22463ef4a8c918d33a7eea9a9e5'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:34.678286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-67ce832f-a458-407b-baff-020e3b6db8a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c93a22463ef4a8c918d33a7eea9a9e5'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:34.696584 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-67ce832f-a458-407b-baff-020e3b6db8a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c93a22463ef4a8c918d33a7eea9a9e5'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:34.697354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-67ce832f-a458-407b-baff-020e3b6db8a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c93a22463ef4a8c918d33a7eea9a9e5'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:34.714595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1351/2701] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:56:34 2022] DELETE /identity/v3/users/3c93a22463ef4a8c918d33a7eea9a9e5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:34.972267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:34.973487 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:34.989978 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2423e-0b95-4ec0-962a-cfc973856984 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:35.081051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e23e8-31c0-4833-a219-6d899e2082b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:35.083022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e23e8-31c0-4833-a219-6d899e2082b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:35.083502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e23e8-31c0-4833-a219-6d899e2082b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:35.083943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e23e8-31c0-4833-a219-6d899e2082b3 admin admin] PATH_INFO: `/v3/projects/153cc4b7aaa94a4d99c5dc9f514c29a5` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:35.090244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e23e8-31c0-4833-a219-6d899e2082b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=153cc4b7aaa94a4d99c5dc9f514c29a5)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:35.093070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e23e8-31c0-4833-a219-6d899e2082b3 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:35.126223 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:35.192411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b83e23e8-31c0-4833-a219-6d899e2082b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '153cc4b7aaa94a4d99c5dc9f514c29a5', 'request_id': 'req-b83e23e8-31c0-4833-a219-6d899e2082b3'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:35.202502 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b83e23e8-31c0-4833-a219-6d899e2082b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '153cc4b7aaa94a4d99c5dc9f514c29a5', 'request_id': 'req-b83e23e8-31c0-4833-a219-6d899e2082b3'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:35.214888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b83e23e8-31c0-4833-a219-6d899e2082b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '153cc4b7aaa94a4d99c5dc9f514c29a5', 'request_id': 'req-b83e23e8-31c0-4833-a219-6d899e2082b3'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:35.241027 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b83e23e8-31c0-4833-a219-6d899e2082b3 admin admin] The token cache is being invalidate because project 153cc4b7aaa94a4d99c5dc9f514c29a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:56:35.241599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b83e23e8-31c0-4833-a219-6d899e2082b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:35.242983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-b83e23e8-31c0-4833-a219-6d899e2082b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:35.246387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1351/2702] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:56:34 2022] DELETE /identity/v3/projects/153cc4b7aaa94a4d99c5dc9f514c29a5 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:35.252638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de32cd-db18-404b-b682-51a0ca0b1aac None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:35.252994 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de32cd-db18-404b-b682-51a0ca0b1aac None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:35.253220 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de32cd-db18-404b-b682-51a0ca0b1aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:35.331433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-96de32cd-db18-404b-b682-51a0ca0b1aac None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:35.333406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-96de32cd-db18-404b-b682-51a0ca0b1aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:35.478397 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1352/2703] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:35.480571 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7817df1-0a75-4463-90b9-70a604e942f3 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:35.480879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7817df1-0a75-4463-90b9-70a604e942f3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:35.481333 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7817df1-0a75-4463-90b9-70a604e942f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:35.523507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-c7817df1-0a75-4463-90b9-70a604e942f3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:35.525490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c7817df1-0a75-4463-90b9-70a604e942f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:35.536350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1352/2704] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:35.540885 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de32cd-db18-404b-b682-51a0ca0b1aac None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:35.553649 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de32cd-db18-404b-b682-51a0ca0b1aac None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:35.559150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f2519-fdd2-49cf-bfb2-b51c09c33d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:35.559547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f2519-fdd2-49cf-bfb2-b51c09c33d47 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:35.560548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f2519-fdd2-49cf-bfb2-b51c09c33d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:35.560752 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f2519-fdd2-49cf-bfb2-b51c09c33d47 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:35.561425 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f2519-fdd2-49cf-bfb2-b51c09c33d47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:56:35.561762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f2519-fdd2-49cf-bfb2-b51c09c33d47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:35.563559 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f2519-fdd2-49cf-bfb2-b51c09c33d47 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:35.594347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7817df1-0a75-4463-90b9-70a604e942f3 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:35.595166 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1353/2705] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:56:35.608307 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7817df1-0a75-4463-90b9-70a604e942f3 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:35.615525 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26b4ba-9005-43e6-b3dd-6cb7c42c1f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:35.616687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26b4ba-9005-43e6-b3dd-6cb7c42c1f3b admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:35.617256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26b4ba-9005-43e6-b3dd-6cb7c42c1f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:35.617820 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26b4ba-9005-43e6-b3dd-6cb7c42c1f3b admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:35.620751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26b4ba-9005-43e6-b3dd-6cb7c42c1f3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:35.625200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26b4ba-9005-43e6-b3dd-6cb7c42c1f3b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:35.678411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1353/2706] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:56:35 2022] POST /identity/v3/projects => generated 359 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:35.686404 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f2519-fdd2-49cf-bfb2-b51c09c33d47 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:35.699851 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f2519-fdd2-49cf-bfb2-b51c09c33d47 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:35.757662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:35.804816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def63e50-f7c0-4840-bd86-78e50c8c3dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:35.805867 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def63e50-f7c0-4840-bd86-78e50c8c3dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:35.806395 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def63e50-f7c0-4840-bd86-78e50c8c3dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:35.807003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def63e50-f7c0-4840-bd86-78e50c8c3dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:35.807987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def63e50-f7c0-4840-bd86-78e50c8c3dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:35.810604 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def63e50-f7c0-4840-bd86-78e50c8c3dd0 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:35.838755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1354/2707] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:56:35 2022] POST /identity/v3/users => generated 280 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:35.842681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26b4ba-9005-43e6-b3dd-6cb7c42c1f3b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:35.860192 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26b4ba-9005-43e6-b3dd-6cb7c42c1f3b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:35.866763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919420c-3dd5-48fe-a470-aa8ea6611472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:35.867720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919420c-3dd5-48fe-a470-aa8ea6611472 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:35.868959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919420c-3dd5-48fe-a470-aa8ea6611472 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:35.870319 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919420c-3dd5-48fe-a470-aa8ea6611472 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:35.872413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919420c-3dd5-48fe-a470-aa8ea6611472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:35.874982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919420c-3dd5-48fe-a470-aa8ea6611472 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:35.889267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1354/2708] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:35.894352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def63e50-f7c0-4840-bd86-78e50c8c3dd0 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:35.914194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def63e50-f7c0-4840-bd86-78e50c8c3dd0 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:35.921771 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58237b63-4d72-4e0c-98a0-d1161a441875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:35.922641 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58237b63-4d72-4e0c-98a0-d1161a441875 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:35.922952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58237b63-4d72-4e0c-98a0-d1161a441875 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:35.923285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58237b63-4d72-4e0c-98a0-d1161a441875 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:35.924000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58237b63-4d72-4e0c-98a0-d1161a441875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:35.926864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58237b63-4d72-4e0c-98a0-d1161a441875 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:35.945381 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1355/2709] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:35.949244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919420c-3dd5-48fe-a470-aa8ea6611472 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:35.967882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919420c-3dd5-48fe-a470-aa8ea6611472 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:35.975076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6d58a-8f4f-4fc5-b5b2-8df58fd4a538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:35.975616 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6d58a-8f4f-4fc5-b5b2-8df58fd4a538 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:35.975754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6d58a-8f4f-4fc5-b5b2-8df58fd4a538 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:35.976015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6d58a-8f4f-4fc5-b5b2-8df58fd4a538 admin admin] PATH_INFO: `/v3/projects/d5d2e9bb8797484ba570171779712bfd/users/2165b2e5a1f04ab5b056744d563456d2/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:36.001368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6d58a-8f4f-4fc5-b5b2-8df58fd4a538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d2e9bb8797484ba570171779712bfd, user_id=2165b2e5a1f04ab5b056744d563456d2, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:36.009146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6d58a-8f4f-4fc5-b5b2-8df58fd4a538 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:36.035356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1355/2710] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:56:35 2022] PUT /identity/v3/projects/d5d2e9bb8797484ba570171779712bfd/users/2165b2e5a1f04ab5b056744d563456d2/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:56:36.046262 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfd123-1cc4-4cb7-bca2-bc7d5cf0cc76 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:36.046500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfd123-1cc4-4cb7-bca2-bc7d5cf0cc76 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:36.046854 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfd123-1cc4-4cb7-bca2-bc7d5cf0cc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:36.087216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-0cbfd123-1cc4-4cb7-bca2-bc7d5cf0cc76 None None] MFA Rules not processed for user `2165b2e5a1f04ab5b056744d563456d2`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:36.088483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0cbfd123-1cc4-4cb7-bca2-bc7d5cf0cc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:36.179498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1356/2711] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:36.215728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6d58a-8f4f-4fc5-b5b2-8df58fd4a538 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:36.217724 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9cc6d58a-8f4f-4fc5-b5b2-8df58fd4a538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:36.320793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6d58a-8f4f-4fc5-b5b2-8df58fd4a538 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:36.324746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0900dd-98f7-49e7-b9d4-aa035505db54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:36.325475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0900dd-98f7-49e7-b9d4-aa035505db54 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:36.325753 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0900dd-98f7-49e7-b9d4-aa035505db54 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:36.326022 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0900dd-98f7-49e7-b9d4-aa035505db54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:36.338924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0900dd-98f7-49e7-b9d4-aa035505db54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:36.340559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0900dd-98f7-49e7-b9d4-aa035505db54 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:36.352122 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1356/2712] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:36.370478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfd123-1cc4-4cb7-bca2-bc7d5cf0cc76 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:36.524518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfd123-1cc4-4cb7-bca2-bc7d5cf0cc76 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:36.539419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:36.539652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:36.539903 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:36.540946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] PATH_INFO: `/v3/users/2165b2e5a1f04ab5b056744d563456d2` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:36.542778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2165b2e5a1f04ab5b056744d563456d2)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:36.553045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:36.679842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:36.687752 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2165b2e5a1f04ab5b056744d563456d2'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:36.699451 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2165b2e5a1f04ab5b056744d563456d2'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:36.700269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2165b2e5a1f04ab5b056744d563456d2'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:36.722030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2165b2e5a1f04ab5b056744d563456d2'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:36.723800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2165b2e5a1f04ab5b056744d563456d2'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:36.738874 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1357/2713] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:56:36 2022] DELETE /identity/v3/users/2165b2e5a1f04ab5b056744d563456d2 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:36.743641 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b62f4-5df7-45e7-83ce-d8b05cf962d8 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:36.743953 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b62f4-5df7-45e7-83ce-d8b05cf962d8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:36.744229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b62f4-5df7-45e7-83ce-d8b05cf962d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:36.770904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-482b62f4-5df7-45e7-83ce-d8b05cf962d8 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:36.773389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-482b62f4-5df7-45e7-83ce-d8b05cf962d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:36.813798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:36.853614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1357/2714] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:36.862969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:36.864952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:36.960275 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab52dd-fe51-47a6-a423-341a17703230 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:36.964912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a50301-c827-48c8-8443-59b402c034e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:36.965535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a50301-c827-48c8-8443-59b402c034e8 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:36.965822 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a50301-c827-48c8-8443-59b402c034e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:36.966095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a50301-c827-48c8-8443-59b402c034e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:36.977180 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a50301-c827-48c8-8443-59b402c034e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:36.978683 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a50301-c827-48c8-8443-59b402c034e8 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:36.990094 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1358/2715] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:37.360126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b62f4-5df7-45e7-83ce-d8b05cf962d8 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:37.361732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-482b62f4-5df7-45e7-83ce-d8b05cf962d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:37.378405 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b62f4-5df7-45e7-83ce-d8b05cf962d8 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:37.392587 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb650b8b-36cd-4c03-b99e-638f6a98022f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:37.393539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb650b8b-36cd-4c03-b99e-638f6a98022f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:37.394015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb650b8b-36cd-4c03-b99e-638f6a98022f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:37.395623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb650b8b-36cd-4c03-b99e-638f6a98022f admin admin] PATH_INFO: `/v3/projects/d5d2e9bb8797484ba570171779712bfd` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:37.397131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb650b8b-36cd-4c03-b99e-638f6a98022f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d2e9bb8797484ba570171779712bfd)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:37.400089 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb650b8b-36cd-4c03-b99e-638f6a98022f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:37.453354 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-cb650b8b-36cd-4c03-b99e-638f6a98022f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d2e9bb8797484ba570171779712bfd', 'request_id': 'req-cb650b8b-36cd-4c03-b99e-638f6a98022f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:37.461642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-cb650b8b-36cd-4c03-b99e-638f6a98022f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d2e9bb8797484ba570171779712bfd', 'request_id': 'req-cb650b8b-36cd-4c03-b99e-638f6a98022f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:37.468470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-cb650b8b-36cd-4c03-b99e-638f6a98022f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d2e9bb8797484ba570171779712bfd', 'request_id': 'req-cb650b8b-36cd-4c03-b99e-638f6a98022f'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:37.480562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-cb650b8b-36cd-4c03-b99e-638f6a98022f admin admin] The token cache is being invalidate because project d5d2e9bb8797484ba570171779712bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:56:37.480851 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-cb650b8b-36cd-4c03-b99e-638f6a98022f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:37.481769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-cb650b8b-36cd-4c03-b99e-638f6a98022f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:37.486579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1358/2716] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:56:37 2022] DELETE /identity/v3/projects/d5d2e9bb8797484ba570171779712bfd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:37.488216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147de752-eacf-48ce-8cbf-a20bbaa56f44 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:37.489058 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147de752-eacf-48ce-8cbf-a20bbaa56f44 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:37.489332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147de752-eacf-48ce-8cbf-a20bbaa56f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:37.511900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-147de752-eacf-48ce-8cbf-a20bbaa56f44 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:37.513800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-147de752-eacf-48ce-8cbf-a20bbaa56f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:37.580550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1359/2717] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:37.582571 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a3bf1-9764-417d-bdde-c99bdfb6848b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:37.582955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a3bf1-9764-417d-bdde-c99bdfb6848b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:37.583574 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a3bf1-9764-417d-bdde-c99bdfb6848b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:37.606887 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-681a3bf1-9764-417d-bdde-c99bdfb6848b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:37.607841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-681a3bf1-9764-417d-bdde-c99bdfb6848b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:37.616696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1359/2718] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:37.623447 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147de752-eacf-48ce-8cbf-a20bbaa56f44 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:37.631694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147de752-eacf-48ce-8cbf-a20bbaa56f44 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:37.636658 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107aa94-ec00-453f-9da1-ae133b0fe025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:37.637289 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107aa94-ec00-453f-9da1-ae133b0fe025 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:37.637555 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107aa94-ec00-453f-9da1-ae133b0fe025 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:37.637900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107aa94-ec00-453f-9da1-ae133b0fe025 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:37.638544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107aa94-ec00-453f-9da1-ae133b0fe025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:56:37.638887 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107aa94-ec00-453f-9da1-ae133b0fe025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:37.640606 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107aa94-ec00-453f-9da1-ae133b0fe025 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:37.652984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1360/2719] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:56:37.657389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a3bf1-9764-417d-bdde-c99bdfb6848b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:37.666998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a3bf1-9764-417d-bdde-c99bdfb6848b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:37.674052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa478fe-283f-487a-90a5-e77af5fa5117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:37.674716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa478fe-283f-487a-90a5-e77af5fa5117 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:37.674932 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa478fe-283f-487a-90a5-e77af5fa5117 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:37.675214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa478fe-283f-487a-90a5-e77af5fa5117 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:37.676448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa478fe-283f-487a-90a5-e77af5fa5117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:37.678565 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa478fe-283f-487a-90a5-e77af5fa5117 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:37.718645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1360/2720] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:56:37 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:37.718997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107aa94-ec00-453f-9da1-ae133b0fe025 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:37.732103 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107aa94-ec00-453f-9da1-ae133b0fe025 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:37.769132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:37.827162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6212072-e072-4aa1-a5c9-1c23a47c1e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:37.827773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6212072-e072-4aa1-a5c9-1c23a47c1e54 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:37.828052 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6212072-e072-4aa1-a5c9-1c23a47c1e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:37.828419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6212072-e072-4aa1-a5c9-1c23a47c1e54 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:37.829068 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6212072-e072-4aa1-a5c9-1c23a47c1e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:37.831038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6212072-e072-4aa1-a5c9-1c23a47c1e54 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:37.861819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1361/2721] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:56:37 2022] POST /identity/v3/users => generated 285 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:56:37.867904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa478fe-283f-487a-90a5-e77af5fa5117 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:37.880800 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa478fe-283f-487a-90a5-e77af5fa5117 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:37.886875 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee558493-829e-4884-8ca0-5c4f68c99ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:37.888214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee558493-829e-4884-8ca0-5c4f68c99ebf admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:37.888505 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee558493-829e-4884-8ca0-5c4f68c99ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:37.888782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee558493-829e-4884-8ca0-5c4f68c99ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:37.889441 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee558493-829e-4884-8ca0-5c4f68c99ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:37.891060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee558493-829e-4884-8ca0-5c4f68c99ebf admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:37.904120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1361/2722] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:37.908896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6212072-e072-4aa1-a5c9-1c23a47c1e54 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:37.922208 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6212072-e072-4aa1-a5c9-1c23a47c1e54 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:37.929998 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d10cd-a94f-44f3-844f-e8b125f7596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:37.930626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d10cd-a94f-44f3-844f-e8b125f7596b admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:37.930906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d10cd-a94f-44f3-844f-e8b125f7596b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:37.931213 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d10cd-a94f-44f3-844f-e8b125f7596b admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:37.931933 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d10cd-a94f-44f3-844f-e8b125f7596b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:37.935422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d10cd-a94f-44f3-844f-e8b125f7596b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:37.953109 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1362/2723] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:37.961009 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee558493-829e-4884-8ca0-5c4f68c99ebf admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:37.972356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee558493-829e-4884-8ca0-5c4f68c99ebf admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:37.977942 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c0fa7-b3de-4f84-a0db-b38628b47728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:37.978689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c0fa7-b3de-4f84-a0db-b38628b47728 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:37.979046 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c0fa7-b3de-4f84-a0db-b38628b47728 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:37.979408 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c0fa7-b3de-4f84-a0db-b38628b47728 admin admin] PATH_INFO: `/v3/projects/3445f064771b48129d64bd65341a9c64/users/b2b08a4997dd4dabaa09404a5ce8f792/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:37.996226 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c0fa7-b3de-4f84-a0db-b38628b47728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3445f064771b48129d64bd65341a9c64, user_id=b2b08a4997dd4dabaa09404a5ce8f792, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:37.999155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c0fa7-b3de-4f84-a0db-b38628b47728 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:38.022135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1362/2724] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:56:37 2022] PUT /identity/v3/projects/3445f064771b48129d64bd65341a9c64/users/b2b08a4997dd4dabaa09404a5ce8f792/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:38.092570 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ef625-05c5-42c2-8a70-8e7c569eccff None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:38.092872 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ef625-05c5-42c2-8a70-8e7c569eccff None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:38.093132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ef625-05c5-42c2-8a70-8e7c569eccff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:38.128599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-a48ef625-05c5-42c2-8a70-8e7c569eccff None None] MFA Rules not processed for user `b2b08a4997dd4dabaa09404a5ce8f792`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:38.131002 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a48ef625-05c5-42c2-8a70-8e7c569eccff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:38.209463 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1363/2725] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:38.221443 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c0fa7-b3de-4f84-a0db-b38628b47728 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:38.225217 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1f9c0fa7-b3de-4f84-a0db-b38628b47728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:38.295311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c0fa7-b3de-4f84-a0db-b38628b47728 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:38.299539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1413b5e-960c-4a87-9094-cdece34836c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06fad9466df14827983dac2b4b844f3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:38.300389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1413b5e-960c-4a87-9094-cdece34836c7 service glance] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:38.300873 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1413b5e-960c-4a87-9094-cdece34836c7 service glance] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:38.301214 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1413b5e-960c-4a87-9094-cdece34836c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:38.315383 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1413b5e-960c-4a87-9094-cdece34836c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:38.315975 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:38.318105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1413b5e-960c-4a87-9094-cdece34836c7 service glance] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:38.341913 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1363/2726] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:41.032297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ef625-05c5-42c2-8a70-8e7c569eccff None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:41.037406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a48ef625-05c5-42c2-8a70-8e7c569eccff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:41.105741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ef625-05c5-42c2-8a70-8e7c569eccff None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:41.109430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f95424-9943-45a0-bf56-1dc86e7a36bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:41.110040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f95424-9943-45a0-bf56-1dc86e7a36bb service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:41.110320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f95424-9943-45a0-bf56-1dc86e7a36bb service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:41.110592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f95424-9943-45a0-bf56-1dc86e7a36bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:41.112534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-63f95424-9943-45a0-bf56-1dc86e7a36bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:41.163539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:41.180430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f95424-9943-45a0-bf56-1dc86e7a36bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:41.182053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f95424-9943-45a0-bf56-1dc86e7a36bb service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:41.193716 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1364/2727] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:44.791579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1413b5e-960c-4a87-9094-cdece34836c7 service glance] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:44.793778 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-f1413b5e-960c-4a87-9094-cdece34836c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:44.811308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1413b5e-960c-4a87-9094-cdece34836c7 service glance] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:44.816109 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d74a1b-f076-4cc2-a7ef-addba1abb7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:44.816793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d74a1b-f076-4cc2-a7ef-addba1abb7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:44.817101 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d74a1b-f076-4cc2-a7ef-addba1abb7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:44.817527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d74a1b-f076-4cc2-a7ef-addba1abb7bd admin admin] PATH_INFO: `/v3/users/18eecdfed8cc44f685095b6a70a4e69d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:44.818756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d74a1b-f076-4cc2-a7ef-addba1abb7bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=18eecdfed8cc44f685095b6a70a4e69d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:44.820720 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d74a1b-f076-4cc2-a7ef-addba1abb7bd admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:44.882347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e5d74a1b-f076-4cc2-a7ef-addba1abb7bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18eecdfed8cc44f685095b6a70a4e69d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:44.914490 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e5d74a1b-f076-4cc2-a7ef-addba1abb7bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18eecdfed8cc44f685095b6a70a4e69d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:44.915284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e5d74a1b-f076-4cc2-a7ef-addba1abb7bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18eecdfed8cc44f685095b6a70a4e69d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:44.915972 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e5d74a1b-f076-4cc2-a7ef-addba1abb7bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18eecdfed8cc44f685095b6a70a4e69d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:44.929370 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e5d74a1b-f076-4cc2-a7ef-addba1abb7bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18eecdfed8cc44f685095b6a70a4e69d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:44.960606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1364/2728] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:56:44 2022] DELETE /identity/v3/users/18eecdfed8cc44f685095b6a70a4e69d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:45.217794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f95424-9943-45a0-bf56-1dc86e7a36bb service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:45.236470 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f95424-9943-45a0-bf56-1dc86e7a36bb service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:45.315266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3056844-8e70-49dc-af7d-36c1954f8b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:45.315917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056844-8e70-49dc-af7d-36c1954f8b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:45.316863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056844-8e70-49dc-af7d-36c1954f8b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:45.317010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056844-8e70-49dc-af7d-36c1954f8b60 admin admin] PATH_INFO: `/v3/projects/40a8edca05bf4490948bb10b390bb373` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:45.318744 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3056844-8e70-49dc-af7d-36c1954f8b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a8edca05bf4490948bb10b390bb373)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:45.321053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3056844-8e70-49dc-af7d-36c1954f8b60 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:45.373655 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f3056844-8e70-49dc-af7d-36c1954f8b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a8edca05bf4490948bb10b390bb373', 'request_id': 'req-f3056844-8e70-49dc-af7d-36c1954f8b60'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:45.384234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f3056844-8e70-49dc-af7d-36c1954f8b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a8edca05bf4490948bb10b390bb373', 'request_id': 'req-f3056844-8e70-49dc-af7d-36c1954f8b60'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:45.390259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f3056844-8e70-49dc-af7d-36c1954f8b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a8edca05bf4490948bb10b390bb373', 'request_id': 'req-f3056844-8e70-49dc-af7d-36c1954f8b60'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:45.404538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f3056844-8e70-49dc-af7d-36c1954f8b60 admin admin] The token cache is being invalidate because project 40a8edca05bf4490948bb10b390bb373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:56:45.404828 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f3056844-8e70-49dc-af7d-36c1954f8b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:45.405601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f3056844-8e70-49dc-af7d-36c1954f8b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:45.407172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1365/2729] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:56:45 2022] DELETE /identity/v3/projects/40a8edca05bf4490948bb10b390bb373 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:45.412166 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad2d5f-f654-4fbe-aa76-f81fb207d780 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:45.412458 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad2d5f-f654-4fbe-aa76-f81fb207d780 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:45.412733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad2d5f-f654-4fbe-aa76-f81fb207d780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:45.436770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-68ad2d5f-f654-4fbe-aa76-f81fb207d780 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:45.438012 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-68ad2d5f-f654-4fbe-aa76-f81fb207d780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:45.521790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1365/2730] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:45.523906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9ab5c-4ead-49ef-9d54-f066dcbd5f05 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:45.524183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9ab5c-4ead-49ef-9d54-f066dcbd5f05 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:45.524450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9ab5c-4ead-49ef-9d54-f066dcbd5f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:45.556665 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-40e9ab5c-4ead-49ef-9d54-f066dcbd5f05 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:45.556979 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-40e9ab5c-4ead-49ef-9d54-f066dcbd5f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:45.566983 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1366/2731] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:45.572196 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad2d5f-f654-4fbe-aa76-f81fb207d780 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:45.583911 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad2d5f-f654-4fbe-aa76-f81fb207d780 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:45.588973 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663654cf-77f8-430f-bf4b-f80e01b557f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:45.589614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663654cf-77f8-430f-bf4b-f80e01b557f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:45.589888 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663654cf-77f8-430f-bf4b-f80e01b557f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:45.590156 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663654cf-77f8-430f-bf4b-f80e01b557f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:45.590816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663654cf-77f8-430f-bf4b-f80e01b557f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:56:45.591114 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663654cf-77f8-430f-bf4b-f80e01b557f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:45.592757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663654cf-77f8-430f-bf4b-f80e01b557f2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:45.606838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1366/2732] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:56:45.611839 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9ab5c-4ead-49ef-9d54-f066dcbd5f05 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:45.622254 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9ab5c-4ead-49ef-9d54-f066dcbd5f05 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:45.627030 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64bc29-4378-4035-82fb-f58c8302063e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:45.627676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64bc29-4378-4035-82fb-f58c8302063e admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:45.627956 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64bc29-4378-4035-82fb-f58c8302063e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:45.628216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64bc29-4378-4035-82fb-f58c8302063e admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:45.629569 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64bc29-4378-4035-82fb-f58c8302063e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:45.631697 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64bc29-4378-4035-82fb-f58c8302063e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:45.667601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1367/2733] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:56:45 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:56:45.676355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663654cf-77f8-430f-bf4b-f80e01b557f2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:45.686593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663654cf-77f8-430f-bf4b-f80e01b557f2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:45.736830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:45.746303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a0b46-04ab-450d-85bf-8a87d4516fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:45.747325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a0b46-04ab-450d-85bf-8a87d4516fec admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:45.747731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a0b46-04ab-450d-85bf-8a87d4516fec admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:45.748174 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a0b46-04ab-450d-85bf-8a87d4516fec admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:45.749026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a0b46-04ab-450d-85bf-8a87d4516fec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:45.751386 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a0b46-04ab-450d-85bf-8a87d4516fec admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:45.775766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1367/2734] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:56:45 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:45.780017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64bc29-4378-4035-82fb-f58c8302063e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:45.794670 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64bc29-4378-4035-82fb-f58c8302063e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:45.799464 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95671751-b86c-4ac0-a9e5-833fc77bfed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:45.800071 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95671751-b86c-4ac0-a9e5-833fc77bfed9 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:45.800345 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95671751-b86c-4ac0-a9e5-833fc77bfed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:45.800611 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95671751-b86c-4ac0-a9e5-833fc77bfed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:45.801259 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95671751-b86c-4ac0-a9e5-833fc77bfed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:45.802893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95671751-b86c-4ac0-a9e5-833fc77bfed9 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:45.813918 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1368/2735] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:45.818525 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a0b46-04ab-450d-85bf-8a87d4516fec admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:45.829115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a0b46-04ab-450d-85bf-8a87d4516fec admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:45.835411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22e985-53e4-4899-a6e7-56964b6c14ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:45.835732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22e985-53e4-4899-a6e7-56964b6c14ca admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:45.836018 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22e985-53e4-4899-a6e7-56964b6c14ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:45.836293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22e985-53e4-4899-a6e7-56964b6c14ca admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:45.836944 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22e985-53e4-4899-a6e7-56964b6c14ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:45.838903 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22e985-53e4-4899-a6e7-56964b6c14ca admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:45.855095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1368/2736] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:45.857441 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95671751-b86c-4ac0-a9e5-833fc77bfed9 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:45.867829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95671751-b86c-4ac0-a9e5-833fc77bfed9 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:45.873009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08137e43-536d-4f01-bf77-3166cdefcba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:45.873731 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08137e43-536d-4f01-bf77-3166cdefcba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:45.874001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08137e43-536d-4f01-bf77-3166cdefcba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:45.874267 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08137e43-536d-4f01-bf77-3166cdefcba3 admin admin] PATH_INFO: `/v3/projects/5653265b717b455f86ac465843c1e221/users/33601cfce42e4120bb39c38c032867e5/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:45.892760 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08137e43-536d-4f01-bf77-3166cdefcba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5653265b717b455f86ac465843c1e221, user_id=33601cfce42e4120bb39c38c032867e5, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:45.896190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08137e43-536d-4f01-bf77-3166cdefcba3 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:45.908675 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1369/2737] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:56:45 2022] PUT /identity/v3/projects/5653265b717b455f86ac465843c1e221/users/33601cfce42e4120bb39c38c032867e5/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:45.921418 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52628c85-131d-43b4-971a-676eb84b6345 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:45.921713 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52628c85-131d-43b4-971a-676eb84b6345 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:45.921980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52628c85-131d-43b4-971a-676eb84b6345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:45.951524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-52628c85-131d-43b4-971a-676eb84b6345 None None] MFA Rules not processed for user `33601cfce42e4120bb39c38c032867e5`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:45.952766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-52628c85-131d-43b4-971a-676eb84b6345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:46.027644 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1369/2738] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:46.032327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08137e43-536d-4f01-bf77-3166cdefcba3 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:46.046244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:46.101282 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08137e43-536d-4f01-bf77-3166cdefcba3 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:46.106662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0ee57-df1d-4009-9e9a-2acddda1f21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:46.107337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0ee57-df1d-4009-9e9a-2acddda1f21b admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:46.107688 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0ee57-df1d-4009-9e9a-2acddda1f21b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:46.108487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0ee57-df1d-4009-9e9a-2acddda1f21b admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:46.108487 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0ee57-df1d-4009-9e9a-2acddda1f21b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:46.110422 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0ee57-df1d-4009-9e9a-2acddda1f21b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:46.145264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1370/2739] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:56:46 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:56:46.146453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52628c85-131d-43b4-971a-676eb84b6345 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:46.158010 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52628c85-131d-43b4-971a-676eb84b6345 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:46.226668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d571e5-627c-41c4-8bfc-2d1410c697d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:46.227449 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d571e5-627c-41c4-8bfc-2d1410c697d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:46.227728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d571e5-627c-41c4-8bfc-2d1410c697d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:46.227998 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d571e5-627c-41c4-8bfc-2d1410c697d4 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:46.228643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d571e5-627c-41c4-8bfc-2d1410c697d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:46.230546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d571e5-627c-41c4-8bfc-2d1410c697d4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:46.252516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1370/2740] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:56:46 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:56:46.257115 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0ee57-df1d-4009-9e9a-2acddda1f21b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:46.267664 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0ee57-df1d-4009-9e9a-2acddda1f21b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:46.273685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc4950-5b75-4418-8475-8cc878c534ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:46.274287 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc4950-5b75-4418-8475-8cc878c534ff admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:46.274564 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc4950-5b75-4418-8475-8cc878c534ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:46.274834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc4950-5b75-4418-8475-8cc878c534ff admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:46.275508 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc4950-5b75-4418-8475-8cc878c534ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:46.277116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc4950-5b75-4418-8475-8cc878c534ff admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:46.288554 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1371/2741] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:46.293267 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d571e5-627c-41c4-8bfc-2d1410c697d4 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:46.303700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d571e5-627c-41c4-8bfc-2d1410c697d4 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:46.309068 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8defb-6787-4621-829a-0e21f70535f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:46.309726 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8defb-6787-4621-829a-0e21f70535f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:46.310005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8defb-6787-4621-829a-0e21f70535f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:46.310275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8defb-6787-4621-829a-0e21f70535f2 admin admin] PATH_INFO: `/v3/projects/6556ea8b5d584087825ddc6b7d28538d/users/44b831d2c7254f3f8d7f668844ca7fd5/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:46.327507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8defb-6787-4621-829a-0e21f70535f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6556ea8b5d584087825ddc6b7d28538d, user_id=44b831d2c7254f3f8d7f668844ca7fd5, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:46.330138 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8defb-6787-4621-829a-0e21f70535f2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:46.345257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1371/2742] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:56:46 2022] PUT /identity/v3/projects/6556ea8b5d584087825ddc6b7d28538d/users/44b831d2c7254f3f8d7f668844ca7fd5/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:56:46.349723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc4950-5b75-4418-8475-8cc878c534ff admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:46.360899 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc4950-5b75-4418-8475-8cc878c534ff admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:46.426805 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4fd54d-323e-4483-b001-8556b75ee6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:46.427518 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fd54d-323e-4483-b001-8556b75ee6de admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:46.427799 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fd54d-323e-4483-b001-8556b75ee6de admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:46.428071 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fd54d-323e-4483-b001-8556b75ee6de admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:46.428710 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4fd54d-323e-4483-b001-8556b75ee6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:46.430313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4fd54d-323e-4483-b001-8556b75ee6de admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:46.444993 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1372/2743] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:46.446192 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8defb-6787-4621-829a-0e21f70535f2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:46.457599 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8defb-6787-4621-829a-0e21f70535f2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:46.462891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99753a-1384-4b40-9d9c-a52f3bac8437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:46.463668 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99753a-1384-4b40-9d9c-a52f3bac8437 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:46.464116 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99753a-1384-4b40-9d9c-a52f3bac8437 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:46.464276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99753a-1384-4b40-9d9c-a52f3bac8437 admin admin] PATH_INFO: `/v3/projects/6556ea8b5d584087825ddc6b7d28538d/users/44b831d2c7254f3f8d7f668844ca7fd5/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:46.466442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad99753a-1384-4b40-9d9c-a52f3bac8437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6556ea8b5d584087825ddc6b7d28538d, user_id=44b831d2c7254f3f8d7f668844ca7fd5, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:46.469278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad99753a-1384-4b40-9d9c-a52f3bac8437 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:46.480598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1372/2744] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:56:46 2022] PUT /identity/v3/projects/6556ea8b5d584087825ddc6b7d28538d/users/44b831d2c7254f3f8d7f668844ca7fd5/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:46.491484 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d61be-e505-420f-9dc6-e64a75d357d8 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:46.491780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d61be-e505-420f-9dc6-e64a75d357d8 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:46.492055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d61be-e505-420f-9dc6-e64a75d357d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:46.521923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-1d6d61be-e505-420f-9dc6-e64a75d357d8 None None] MFA Rules not processed for user `44b831d2c7254f3f8d7f668844ca7fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:46.523177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1d6d61be-e505-420f-9dc6-e64a75d357d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:46.594693 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1373/2745] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:56:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:46.606112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99753a-1384-4b40-9d9c-a52f3bac8437 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:46.607818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ad99753a-1384-4b40-9d9c-a52f3bac8437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:46.630413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:46.696272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99753a-1384-4b40-9d9c-a52f3bac8437 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:46.700373 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd0ee0-d9cb-417b-bec9-dda95cd7ffd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:46.701043 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0ee0-d9cb-417b-bec9-dda95cd7ffd7 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:46.701323 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0ee0-d9cb-417b-bec9-dda95cd7ffd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:46.701592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0ee0-d9cb-417b-bec9-dda95cd7ffd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:46.711370 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd0ee0-d9cb-417b-bec9-dda95cd7ffd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:46.712817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd0ee0-d9cb-417b-bec9-dda95cd7ffd7 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:46.723021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1373/2746] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:46.803913 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d61be-e505-420f-9dc6-e64a75d357d8 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:46.815474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d61be-e505-420f-9dc6-e64a75d357d8 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:46.819062 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccddc8-ccd3-4e74-b1a9-b8dd1dceba1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:46.819690 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccddc8-ccd3-4e74-b1a9-b8dd1dceba1f service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:46.819986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccddc8-ccd3-4e74-b1a9-b8dd1dceba1f service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:46.820231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccddc8-ccd3-4e74-b1a9-b8dd1dceba1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:46.825438 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:46.882415 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccddc8-ccd3-4e74-b1a9-b8dd1dceba1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:46.883813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccddc8-ccd3-4e74-b1a9-b8dd1dceba1f service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:46.892662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1374/2747] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:56.718217 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=28 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:56.888560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=24 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:01.170199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd0ee0-d9cb-417b-bec9-dda95cd7ffd7 service cinder] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:01.237983 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd0ee0-d9cb-417b-bec9-dda95cd7ffd7 service cinder] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:01.242698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:01.243352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:01.243633 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:01.244170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] PATH_INFO: `/v3/users/33601cfce42e4120bb39c38c032867e5` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:01.245284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33601cfce42e4120bb39c38c032867e5)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:01.247217 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:01.294712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33601cfce42e4120bb39c38c032867e5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:01.307520 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33601cfce42e4120bb39c38c032867e5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:01.308243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33601cfce42e4120bb39c38c032867e5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:01.308863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33601cfce42e4120bb39c38c032867e5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:01.316810 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33601cfce42e4120bb39c38c032867e5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:01.330858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1374/2748] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:01 2022] DELETE /identity/v3/users/33601cfce42e4120bb39c38c032867e5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:01.335399 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fa877-c234-4075-9b68-ea7fe7c558ec None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:01.335957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fa877-c234-4075-9b68-ea7fe7c558ec None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:01.335957 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fa877-c234-4075-9b68-ea7fe7c558ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:01.355623 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-7d7fa877-c234-4075-9b68-ea7fe7c558ec None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:01.356848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7d7fa877-c234-4075-9b68-ea7fe7c558ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:01.415248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1375/2749] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:01.425348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:01.426777 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:01.485961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6afb1a-1585-4ba1-923c-c43d455f9687 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:01.489603 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95373b-6570-4f5a-a2e1-61afcabdfab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:01.490199 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95373b-6570-4f5a-a2e1-61afcabdfab3 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:01.490477 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95373b-6570-4f5a-a2e1-61afcabdfab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:01.490746 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95373b-6570-4f5a-a2e1-61afcabdfab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:01.501781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95373b-6570-4f5a-a2e1-61afcabdfab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:01.503075 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95373b-6570-4f5a-a2e1-61afcabdfab3 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:01.514725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1375/2750] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:01.815930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fa877-c234-4075-9b68-ea7fe7c558ec None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:01.817512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7d7fa877-c234-4075-9b68-ea7fe7c558ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:01.836076 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fa877-c234-4075-9b68-ea7fe7c558ec None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:01.841049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:01.842502 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:01.842771 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:01.843203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] PATH_INFO: `/v3/projects/5653265b717b455f86ac465843c1e221` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:01.845437 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5653265b717b455f86ac465843c1e221)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:01.847820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:01.909539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5653265b717b455f86ac465843c1e221', 'request_id': 'req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:01.917758 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5653265b717b455f86ac465843c1e221', 'request_id': 'req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:01.924174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5653265b717b455f86ac465843c1e221', 'request_id': 'req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:01.938464 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] The token cache is being invalidate because project 5653265b717b455f86ac465843c1e221 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:01.938970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:01.939678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:01.941468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1376/2751] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:01 2022] DELETE /identity/v3/projects/5653265b717b455f86ac465843c1e221 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:57:01.945838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95373b-6570-4f5a-a2e1-61afcabdfab3 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:01.955798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95373b-6570-4f5a-a2e1-61afcabdfab3 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:01.979336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:02.026419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:02.027068 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:02.027387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:02.027657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] PATH_INFO: `/v3/users/44b831d2c7254f3f8d7f668844ca7fd5` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:02.028829 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b831d2c7254f3f8d7f668844ca7fd5)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:02.030748 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:02.080858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b831d2c7254f3f8d7f668844ca7fd5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.095316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b831d2c7254f3f8d7f668844ca7fd5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.096074 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b831d2c7254f3f8d7f668844ca7fd5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.096703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b831d2c7254f3f8d7f668844ca7fd5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.106752 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b831d2c7254f3f8d7f668844ca7fd5'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.124438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1376/2752] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:01 2022] DELETE /identity/v3/users/44b831d2c7254f3f8d7f668844ca7fd5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:57:02.161654 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:02.163193 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:02.261133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a371f-4392-4481-9dfe-b3d35ce0a5a6 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:02.308929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:02.327037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e744d-3864-4325-ae7e-75fcf0a6375a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:02.327964 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e744d-3864-4325-ae7e-75fcf0a6375a service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:02.328244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e744d-3864-4325-ae7e-75fcf0a6375a service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:02.328516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e744d-3864-4325-ae7e-75fcf0a6375a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:02.330281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-605e744d-3864-4325-ae7e-75fcf0a6375a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:02.412862 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e744d-3864-4325-ae7e-75fcf0a6375a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:02.415033 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e744d-3864-4325-ae7e-75fcf0a6375a service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:02.431443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1377/2753] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:02.519653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:02.522045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:02.670210 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823ad55-0b91-4f71-86f4-42c26b4b919f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:02.678127 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d27ada-8bbb-4d5c-ac63-85731b1c4988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:02.678858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d27ada-8bbb-4d5c-ac63-85731b1c4988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:02.679328 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d27ada-8bbb-4d5c-ac63-85731b1c4988 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:02.679694 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d27ada-8bbb-4d5c-ac63-85731b1c4988 admin admin] PATH_INFO: `/v3/projects/6556ea8b5d584087825ddc6b7d28538d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:02.681303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d27ada-8bbb-4d5c-ac63-85731b1c4988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6556ea8b5d584087825ddc6b7d28538d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:02.684698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d27ada-8bbb-4d5c-ac63-85731b1c4988 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:02.755838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:02.766891 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-63d27ada-8bbb-4d5c-ac63-85731b1c4988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6556ea8b5d584087825ddc6b7d28538d', 'request_id': 'req-63d27ada-8bbb-4d5c-ac63-85731b1c4988'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.781844 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-63d27ada-8bbb-4d5c-ac63-85731b1c4988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6556ea8b5d584087825ddc6b7d28538d', 'request_id': 'req-63d27ada-8bbb-4d5c-ac63-85731b1c4988'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.789108 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-63d27ada-8bbb-4d5c-ac63-85731b1c4988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6556ea8b5d584087825ddc6b7d28538d', 'request_id': 'req-63d27ada-8bbb-4d5c-ac63-85731b1c4988'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.804325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-63d27ada-8bbb-4d5c-ac63-85731b1c4988 admin admin] The token cache is being invalidate because project 6556ea8b5d584087825ddc6b7d28538d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:02.806721 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-63d27ada-8bbb-4d5c-ac63-85731b1c4988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.807212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-63d27ada-8bbb-4d5c-ac63-85731b1c4988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.809284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1377/2754] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:02 2022] DELETE /identity/v3/projects/6556ea8b5d584087825ddc6b7d28538d => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:02.814086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb837a-788e-42de-8f99-9387fea3c59c None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:02.814653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb837a-788e-42de-8f99-9387fea3c59c None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:02.814932 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb837a-788e-42de-8f99-9387fea3c59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:02.837244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-efbb837a-788e-42de-8f99-9387fea3c59c None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:02.838911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-efbb837a-788e-42de-8f99-9387fea3c59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:02.915599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1378/2755] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:02.923347 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a89b23-fe24-47c9-82b6-878d3fe53da3 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:02.923635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a89b23-fe24-47c9-82b6-878d3fe53da3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:02.923907 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a89b23-fe24-47c9-82b6-878d3fe53da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:02.947135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-62a89b23-fe24-47c9-82b6-878d3fe53da3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:02.948381 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-62a89b23-fe24-47c9-82b6-878d3fe53da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:02.957302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1378/2756] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:02.962699 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb837a-788e-42de-8f99-9387fea3c59c None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:02.972953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb837a-788e-42de-8f99-9387fea3c59c None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:02.980260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258b5be-9c67-46e0-998c-719f09bca6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:02.980880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258b5be-9c67-46e0-998c-719f09bca6cd admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:02.981156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258b5be-9c67-46e0-998c-719f09bca6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:02.981418 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258b5be-9c67-46e0-998c-719f09bca6cd admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:02.982067 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258b5be-9c67-46e0-998c-719f09bca6cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:57:02.982359 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258b5be-9c67-46e0-998c-719f09bca6cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:02.984000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258b5be-9c67-46e0-998c-719f09bca6cd admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:02.997915 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1379/2757] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:57:03.003330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a89b23-fe24-47c9-82b6-878d3fe53da3 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.016212 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a89b23-fe24-47c9-82b6-878d3fe53da3 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.022657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb950a-fbcc-410a-97b7-c246d72f866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.023955 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb950a-fbcc-410a-97b7-c246d72f866f admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.024420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb950a-fbcc-410a-97b7-c246d72f866f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.024773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb950a-fbcc-410a-97b7-c246d72f866f admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.025517 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb950a-fbcc-410a-97b7-c246d72f866f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.027675 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb950a-fbcc-410a-97b7-c246d72f866f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.067252 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1379/2758] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/projects => generated 357 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:03.068867 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258b5be-9c67-46e0-998c-719f09bca6cd admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.082079 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258b5be-9c67-46e0-998c-719f09bca6cd admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.156219 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f6e1c-a99f-42e8-8c3c-6f1cd8bf33cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.157022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f6e1c-a99f-42e8-8c3c-6f1cd8bf33cc admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.157311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f6e1c-a99f-42e8-8c3c-6f1cd8bf33cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.157556 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f6e1c-a99f-42e8-8c3c-6f1cd8bf33cc admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.158248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f6e1c-a99f-42e8-8c3c-6f1cd8bf33cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.160234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f6e1c-a99f-42e8-8c3c-6f1cd8bf33cc admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.186527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1380/2759] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/users => generated 279 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:03.190867 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb950a-fbcc-410a-97b7-c246d72f866f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.203194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb950a-fbcc-410a-97b7-c246d72f866f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.207962 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa421ba-1f34-4dbe-a8e3-9ed87085630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.208598 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa421ba-1f34-4dbe-a8e3-9ed87085630f admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.208859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa421ba-1f34-4dbe-a8e3-9ed87085630f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.209135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa421ba-1f34-4dbe-a8e3-9ed87085630f admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.209784 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa421ba-1f34-4dbe-a8e3-9ed87085630f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.212006 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa421ba-1f34-4dbe-a8e3-9ed87085630f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.233186 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1380/2760] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:03.234408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f6e1c-a99f-42e8-8c3c-6f1cd8bf33cc admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.244740 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f6e1c-a99f-42e8-8c3c-6f1cd8bf33cc admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.251110 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be0abc-939e-4e5e-bb4b-c46727636535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.251810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be0abc-939e-4e5e-bb4b-c46727636535 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.252022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be0abc-939e-4e5e-bb4b-c46727636535 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.252294 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be0abc-939e-4e5e-bb4b-c46727636535 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.252939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be0abc-939e-4e5e-bb4b-c46727636535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.254529 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be0abc-939e-4e5e-bb4b-c46727636535 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.256285 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:03.270201 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1381/2761] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:03.275276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa421ba-1f34-4dbe-a8e3-9ed87085630f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.285627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa421ba-1f34-4dbe-a8e3-9ed87085630f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.290633 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054ac55-791e-4059-90aa-f6b114000a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.291494 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054ac55-791e-4059-90aa-f6b114000a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.291773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054ac55-791e-4059-90aa-f6b114000a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.292045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054ac55-791e-4059-90aa-f6b114000a72 admin admin] PATH_INFO: `/v3/projects/942cd0c15b1449edb4c9734788a3afc1/users/01b7dddea2f0412cb323a2be0d0a4abe/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.307786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054ac55-791e-4059-90aa-f6b114000a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=942cd0c15b1449edb4c9734788a3afc1, user_id=01b7dddea2f0412cb323a2be0d0a4abe, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.310448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054ac55-791e-4059-90aa-f6b114000a72 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.327019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1381/2762] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:03 2022] PUT /identity/v3/projects/942cd0c15b1449edb4c9734788a3afc1/users/01b7dddea2f0412cb323a2be0d0a4abe/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:57:03.339870 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71dc191-05de-4d3d-8cdc-f23b24e94311 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.340175 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71dc191-05de-4d3d-8cdc-f23b24e94311 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.340402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71dc191-05de-4d3d-8cdc-f23b24e94311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.371916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b71dc191-05de-4d3d-8cdc-f23b24e94311 None None] MFA Rules not processed for user `01b7dddea2f0412cb323a2be0d0a4abe`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:03.373206 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b71dc191-05de-4d3d-8cdc-f23b24e94311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:03.441863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1382/2763] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:03.448285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054ac55-791e-4059-90aa-f6b114000a72 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.514125 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054ac55-791e-4059-90aa-f6b114000a72 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.518833 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9dce4-8e30-4e33-aac8-8430c28d0ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.519499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9dce4-8e30-4e33-aac8-8430c28d0ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.519783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9dce4-8e30-4e33-aac8-8430c28d0ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.520353 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9dce4-8e30-4e33-aac8-8430c28d0ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.520684 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9dce4-8e30-4e33-aac8-8430c28d0ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.522733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9dce4-8e30-4e33-aac8-8430c28d0ca1 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.551046 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1382/2764] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/projects => generated 355 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:03.555248 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71dc191-05de-4d3d-8cdc-f23b24e94311 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.566390 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71dc191-05de-4d3d-8cdc-f23b24e94311 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.627970 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90acef-3332-4279-ad41-61ae7ae4af86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.628696 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90acef-3332-4279-ad41-61ae7ae4af86 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.629150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90acef-3332-4279-ad41-61ae7ae4af86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.629401 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90acef-3332-4279-ad41-61ae7ae4af86 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.630098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90acef-3332-4279-ad41-61ae7ae4af86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.632140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90acef-3332-4279-ad41-61ae7ae4af86 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.653846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1383/2765] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:57:03.657816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9dce4-8e30-4e33-aac8-8430c28d0ca1 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.669161 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9dce4-8e30-4e33-aac8-8430c28d0ca1 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.673816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95462fb-397b-461c-84e2-1876387ace11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.674424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95462fb-397b-461c-84e2-1876387ace11 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.674708 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95462fb-397b-461c-84e2-1876387ace11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.674984 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95462fb-397b-461c-84e2-1876387ace11 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.675885 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95462fb-397b-461c-84e2-1876387ace11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.677481 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95462fb-397b-461c-84e2-1876387ace11 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.688660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1383/2766] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:03.692825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90acef-3332-4279-ad41-61ae7ae4af86 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.703029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90acef-3332-4279-ad41-61ae7ae4af86 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.708237 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6e59f-67fc-4704-a463-5fa448f2a48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.709085 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6e59f-67fc-4704-a463-5fa448f2a48f admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.709495 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6e59f-67fc-4704-a463-5fa448f2a48f admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.709883 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6e59f-67fc-4704-a463-5fa448f2a48f admin admin] PATH_INFO: `/v3/projects/1cea3999cda44b63a768019c967eac86/users/13eb11e8b3c54eb098aa8265a2309a6e/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.726191 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6e59f-67fc-4704-a463-5fa448f2a48f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cea3999cda44b63a768019c967eac86, user_id=13eb11e8b3c54eb098aa8265a2309a6e, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.729516 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6e59f-67fc-4704-a463-5fa448f2a48f admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.744008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1384/2767] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:03 2022] PUT /identity/v3/projects/1cea3999cda44b63a768019c967eac86/users/13eb11e8b3c54eb098aa8265a2309a6e/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:03.748092 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95462fb-397b-461c-84e2-1876387ace11 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.762205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95462fb-397b-461c-84e2-1876387ace11 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.824327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:03.841007 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e586d4d-950d-41a6-ac34-8384ddcbdc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.841615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e586d4d-950d-41a6-ac34-8384ddcbdc16 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.842021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e586d4d-950d-41a6-ac34-8384ddcbdc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.842296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e586d4d-950d-41a6-ac34-8384ddcbdc16 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.842940 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e586d4d-950d-41a6-ac34-8384ddcbdc16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.844635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e586d4d-950d-41a6-ac34-8384ddcbdc16 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.855819 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1384/2768] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:57:03.860204 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6e59f-67fc-4704-a463-5fa448f2a48f admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.872280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6e59f-67fc-4704-a463-5fa448f2a48f admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.876977 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f083a-4690-4442-83e7-9bc23d34e7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.877627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f083a-4690-4442-83e7-9bc23d34e7bb admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.877898 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f083a-4690-4442-83e7-9bc23d34e7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.878163 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f083a-4690-4442-83e7-9bc23d34e7bb admin admin] PATH_INFO: `/v3/projects/1cea3999cda44b63a768019c967eac86/users/13eb11e8b3c54eb098aa8265a2309a6e/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.880089 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f083a-4690-4442-83e7-9bc23d34e7bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cea3999cda44b63a768019c967eac86, user_id=13eb11e8b3c54eb098aa8265a2309a6e, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.882723 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f083a-4690-4442-83e7-9bc23d34e7bb admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.894859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1385/2769] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:03 2022] PUT /identity/v3/projects/1cea3999cda44b63a768019c967eac86/users/13eb11e8b3c54eb098aa8265a2309a6e/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:03.905966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0d89a-fe50-47ce-8812-4e31640231f8 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.906256 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0d89a-fe50-47ce-8812-4e31640231f8 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.906534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0d89a-fe50-47ce-8812-4e31640231f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.942949 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-8fc0d89a-fe50-47ce-8812-4e31640231f8 None None] MFA Rules not processed for user `13eb11e8b3c54eb098aa8265a2309a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:03.944334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-8fc0d89a-fe50-47ce-8812-4e31640231f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:04.029342 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1385/2770] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:04.041877 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f083a-4690-4442-83e7-9bc23d34e7bb admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:04.043854 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-830f083a-4690-4442-83e7-9bc23d34e7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:04.112762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:04.130815 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f083a-4690-4442-83e7-9bc23d34e7bb admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:04.135937 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf057f-28ec-448c-9ce8-ccb31039948d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:04.136551 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf057f-28ec-448c-9ce8-ccb31039948d service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:04.136825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf057f-28ec-448c-9ce8-ccb31039948d service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:04.137095 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf057f-28ec-448c-9ce8-ccb31039948d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:04.146897 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf057f-28ec-448c-9ce8-ccb31039948d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:04.149080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf057f-28ec-448c-9ce8-ccb31039948d service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:04.159158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1386/2771] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:04.239818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0d89a-fe50-47ce-8812-4e31640231f8 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:04.251021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0d89a-fe50-47ce-8812-4e31640231f8 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:04.254706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10d43e-4f7d-480f-ad31-4037b4b0a90e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:04.255350 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10d43e-4f7d-480f-ad31-4037b4b0a90e service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:04.255635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10d43e-4f7d-480f-ad31-4037b4b0a90e service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:04.256064 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10d43e-4f7d-480f-ad31-4037b4b0a90e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:04.315805 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10d43e-4f7d-480f-ad31-4037b4b0a90e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:04.317244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10d43e-4f7d-480f-ad31-4037b4b0a90e service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:04.327966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1386/2772] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:07.923909 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf057f-28ec-448c-9ce8-ccb31039948d service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:07.925404 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-abaf057f-28ec-448c-9ce8-ccb31039948d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:08.080514 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf057f-28ec-448c-9ce8-ccb31039948d service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:08.092373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1edc6-7e70-4935-85c7-e5f5fc7bea47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:08.093536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1edc6-7e70-4935-85c7-e5f5fc7bea47 service placement] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:08.094131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1edc6-7e70-4935-85c7-e5f5fc7bea47 service placement] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:08.094663 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1edc6-7e70-4935-85c7-e5f5fc7bea47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:08.097533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0bc1edc6-7e70-4935-85c7-e5f5fc7bea47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:08.112853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1edc6-7e70-4935-85c7-e5f5fc7bea47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:08.114482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1edc6-7e70-4935-85c7-e5f5fc7bea47 service placement] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:08.127829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1387/2773] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:14.323585 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=72 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:18.122633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=50 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:21.842058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30770a3-0303-4ea8-9198-fef8d72b0532 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:21.842634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30770a3-0303-4ea8-9198-fef8d72b0532 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:21.842634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30770a3-0303-4ea8-9198-fef8d72b0532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:21.861918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-a30770a3-0303-4ea8-9198-fef8d72b0532 None None] MFA Rules not processed for user `488e41d766134f3aa7c956816d01df06`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:21.863149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a30770a3-0303-4ea8-9198-fef8d72b0532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:21.919107 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1387/2774] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:24.976946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1edc6-7e70-4935-85c7-e5f5fc7bea47 service placement] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:24.978864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0bc1edc6-7e70-4935-85c7-e5f5fc7bea47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:25.054698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1edc6-7e70-4935-85c7-e5f5fc7bea47 service placement] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:25.062170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:25.062669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:25.062960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:25.063235 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] PATH_INFO: `/v3/users/b2b08a4997dd4dabaa09404a5ce8f792` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:25.064439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b08a4997dd4dabaa09404a5ce8f792)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:25.066574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:25.125162 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b08a4997dd4dabaa09404a5ce8f792'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:25.134433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b08a4997dd4dabaa09404a5ce8f792'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:25.135041 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b08a4997dd4dabaa09404a5ce8f792'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:25.151488 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b08a4997dd4dabaa09404a5ce8f792'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:25.152634 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b08a4997dd4dabaa09404a5ce8f792'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:25.168782 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1388/2775] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:24 2022] DELETE /identity/v3/users/b2b08a4997dd4dabaa09404a5ce8f792 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:25.173734 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a924678-fc82-43ea-9cb2-048c9c7ed40b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:25.174700 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a924678-fc82-43ea-9cb2-048c9c7ed40b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:25.175005 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a924678-fc82-43ea-9cb2-048c9c7ed40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:25.211234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-1a924678-fc82-43ea-9cb2-048c9c7ed40b None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:25.214263 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1a924678-fc82-43ea-9cb2-048c9c7ed40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:25.284116 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1388/2776] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:25.294891 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:25.296478 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:25.355345 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd3451-fd83-4636-805c-69ba88975c61 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:25.359055 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b60b17-8e7b-460e-a22b-768ffe9ca425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:25.359694 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b60b17-8e7b-460e-a22b-768ffe9ca425 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:25.359973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b60b17-8e7b-460e-a22b-768ffe9ca425 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:25.360244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b60b17-8e7b-460e-a22b-768ffe9ca425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:25.370371 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b60b17-8e7b-460e-a22b-768ffe9ca425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:25.371881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b60b17-8e7b-460e-a22b-768ffe9ca425 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:25.381596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1389/2777] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:57:25.666575 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a924678-fc82-43ea-9cb2-048c9c7ed40b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:25.668139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1a924678-fc82-43ea-9cb2-048c9c7ed40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:25.681534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a924678-fc82-43ea-9cb2-048c9c7ed40b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:25.698305 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e21b04-ae0a-4d43-b243-e12f9bf14714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:25.699087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e21b04-ae0a-4d43-b243-e12f9bf14714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:25.699635 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e21b04-ae0a-4d43-b243-e12f9bf14714 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:25.699969 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e21b04-ae0a-4d43-b243-e12f9bf14714 admin admin] PATH_INFO: `/v3/projects/3445f064771b48129d64bd65341a9c64` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:25.701678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e21b04-ae0a-4d43-b243-e12f9bf14714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3445f064771b48129d64bd65341a9c64)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:25.703935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e21b04-ae0a-4d43-b243-e12f9bf14714 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:25.751904 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-17e21b04-ae0a-4d43-b243-e12f9bf14714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3445f064771b48129d64bd65341a9c64', 'request_id': 'req-17e21b04-ae0a-4d43-b243-e12f9bf14714'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:25.759853 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-17e21b04-ae0a-4d43-b243-e12f9bf14714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3445f064771b48129d64bd65341a9c64', 'request_id': 'req-17e21b04-ae0a-4d43-b243-e12f9bf14714'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:25.766296 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-17e21b04-ae0a-4d43-b243-e12f9bf14714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3445f064771b48129d64bd65341a9c64', 'request_id': 'req-17e21b04-ae0a-4d43-b243-e12f9bf14714'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:25.777769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-17e21b04-ae0a-4d43-b243-e12f9bf14714 admin admin] The token cache is being invalidate because project 3445f064771b48129d64bd65341a9c64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:25.778058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-17e21b04-ae0a-4d43-b243-e12f9bf14714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:25.778693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-17e21b04-ae0a-4d43-b243-e12f9bf14714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:25.780253 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1389/2778] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:25 2022] DELETE /identity/v3/projects/3445f064771b48129d64bd65341a9c64 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:25.785176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f85558-f2df-465f-a644-7631edeff72a None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:25.785508 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f85558-f2df-465f-a644-7631edeff72a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:25.785785 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f85558-f2df-465f-a644-7631edeff72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:25.808704 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-06f85558-f2df-465f-a644-7631edeff72a None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:25.809892 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-06f85558-f2df-465f-a644-7631edeff72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:25.827517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:25.872435 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1390/2779] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:25.880816 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c093a-3862-41f2-9ea6-b036fa8a85b7 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:25.881105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c093a-3862-41f2-9ea6-b036fa8a85b7 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:25.881388 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c093a-3862-41f2-9ea6-b036fa8a85b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:25.903516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-ce9c093a-3862-41f2-9ea6-b036fa8a85b7 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:25.904753 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-ce9c093a-3862-41f2-9ea6-b036fa8a85b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:25.912890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1390/2780] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:25.917523 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f85558-f2df-465f-a644-7631edeff72a None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:25.927528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f85558-f2df-465f-a644-7631edeff72a None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:25.933482 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62baf0-156f-45ac-abf3-71acf4d4d026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:25.934093 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62baf0-156f-45ac-abf3-71acf4d4d026 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:25.934369 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62baf0-156f-45ac-abf3-71acf4d4d026 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:25.934630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62baf0-156f-45ac-abf3-71acf4d4d026 admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:25.935298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62baf0-156f-45ac-abf3-71acf4d4d026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:57:25.935596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62baf0-156f-45ac-abf3-71acf4d4d026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:25.937192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62baf0-156f-45ac-abf3-71acf4d4d026 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:25.949883 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1391/2781] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:57:25.954228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c093a-3862-41f2-9ea6-b036fa8a85b7 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:25.963521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c093a-3862-41f2-9ea6-b036fa8a85b7 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:25.968790 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e1467-0e69-4576-bbda-81b0fb7e257e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:25.969382 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e1467-0e69-4576-bbda-81b0fb7e257e admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:25.969657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e1467-0e69-4576-bbda-81b0fb7e257e admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:25.970216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e1467-0e69-4576-bbda-81b0fb7e257e admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:25.970542 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e1467-0e69-4576-bbda-81b0fb7e257e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:25.973056 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e1467-0e69-4576-bbda-81b0fb7e257e admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:25.998097 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:26.006522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1391/2782] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:25 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:26.012126 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62baf0-156f-45ac-abf3-71acf4d4d026 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:26.022429 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62baf0-156f-45ac-abf3-71acf4d4d026 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:26.082158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b661ac-d11a-4fb9-a890-0bd9b4cbb82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:26.082810 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b661ac-d11a-4fb9-a890-0bd9b4cbb82b admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:26.083090 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b661ac-d11a-4fb9-a890-0bd9b4cbb82b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:26.083394 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b661ac-d11a-4fb9-a890-0bd9b4cbb82b admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:26.084026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b661ac-d11a-4fb9-a890-0bd9b4cbb82b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:26.086170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b661ac-d11a-4fb9-a890-0bd9b4cbb82b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:26.109695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1392/2783] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:26 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:26.113793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e1467-0e69-4576-bbda-81b0fb7e257e admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:26.125581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e1467-0e69-4576-bbda-81b0fb7e257e admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:26.130389 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3931b32-3d6f-408e-aa37-3ca304ecb861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:26.131051 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3931b32-3d6f-408e-aa37-3ca304ecb861 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:26.131370 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3931b32-3d6f-408e-aa37-3ca304ecb861 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:26.131727 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3931b32-3d6f-408e-aa37-3ca304ecb861 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:26.132579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3931b32-3d6f-408e-aa37-3ca304ecb861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:26.134170 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3931b32-3d6f-408e-aa37-3ca304ecb861 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:26.144348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1392/2784] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:26.148974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b661ac-d11a-4fb9-a890-0bd9b4cbb82b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:26.158819 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b661ac-d11a-4fb9-a890-0bd9b4cbb82b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:26.164472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d7141-91c7-4c5b-a1dc-38ff46b29f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:26.165075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d7141-91c7-4c5b-a1dc-38ff46b29f3d admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:26.165354 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d7141-91c7-4c5b-a1dc-38ff46b29f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:26.165626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d7141-91c7-4c5b-a1dc-38ff46b29f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:26.166261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d7141-91c7-4c5b-a1dc-38ff46b29f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:26.167893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d7141-91c7-4c5b-a1dc-38ff46b29f3d admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:26.178542 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1393/2785] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:26.183890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3931b32-3d6f-408e-aa37-3ca304ecb861 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:26.195284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3931b32-3d6f-408e-aa37-3ca304ecb861 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:26.200129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02970eb5-48e5-475f-a953-7a702815898c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:26.200787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02970eb5-48e5-475f-a953-7a702815898c admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:26.201070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02970eb5-48e5-475f-a953-7a702815898c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:26.201345 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02970eb5-48e5-475f-a953-7a702815898c admin admin] PATH_INFO: `/v3/projects/e73cfdbb50144b67a614836f73ed5b3d/users/f4003700359449afb83a7352ff2b326a/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:26.217292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02970eb5-48e5-475f-a953-7a702815898c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e73cfdbb50144b67a614836f73ed5b3d, user_id=f4003700359449afb83a7352ff2b326a, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:26.219977 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02970eb5-48e5-475f-a953-7a702815898c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:26.234228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1393/2786] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:26 2022] PUT /identity/v3/projects/e73cfdbb50144b67a614836f73ed5b3d/users/f4003700359449afb83a7352ff2b326a/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:26.239260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02877be-f70c-458b-8ea2-3a7703976d87 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:26.239549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02877be-f70c-458b-8ea2-3a7703976d87 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:26.239821 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02877be-f70c-458b-8ea2-3a7703976d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:26.262019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f02877be-f70c-458b-8ea2-3a7703976d87 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:26.263607 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f02877be-f70c-458b-8ea2-3a7703976d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:26.327821 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1394/2787] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:57:26.338536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02970eb5-48e5-475f-a953-7a702815898c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:26.340278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-02970eb5-48e5-475f-a953-7a702815898c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:26.407788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02970eb5-48e5-475f-a953-7a702815898c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:26.411456 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134be220-07e9-4142-b575-d334fa06faf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:26.412055 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134be220-07e9-4142-b575-d334fa06faf4 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:26.412336 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134be220-07e9-4142-b575-d334fa06faf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:26.412606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134be220-07e9-4142-b575-d334fa06faf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:26.423030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134be220-07e9-4142-b575-d334fa06faf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:26.424648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134be220-07e9-4142-b575-d334fa06faf4 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:26.435069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1394/2788] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:28.222635 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02877be-f70c-458b-8ea2-3a7703976d87 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:28.224151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f02877be-f70c-458b-8ea2-3a7703976d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:28.237627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02877be-f70c-458b-8ea2-3a7703976d87 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:28.243780 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:28.244715 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:28.245188 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:28.245633 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] PATH_INFO: `/v3/users/01b7dddea2f0412cb323a2be0d0a4abe` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:28.247829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01b7dddea2f0412cb323a2be0d0a4abe)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:28.250075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:28.291236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:28.328826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01b7dddea2f0412cb323a2be0d0a4abe'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:28.337574 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01b7dddea2f0412cb323a2be0d0a4abe'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:28.338344 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01b7dddea2f0412cb323a2be0d0a4abe'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:28.352145 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01b7dddea2f0412cb323a2be0d0a4abe'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:28.352974 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01b7dddea2f0412cb323a2be0d0a4abe'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:28.367748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1395/2789] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:28 2022] DELETE /identity/v3/users/01b7dddea2f0412cb323a2be0d0a4abe => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:28.373154 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01638e6e-276d-4211-9f7c-8afc8d4f484f None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:28.373687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01638e6e-276d-4211-9f7c-8afc8d4f484f None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:28.373687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01638e6e-276d-4211-9f7c-8afc8d4f484f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:28.394711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-01638e6e-276d-4211-9f7c-8afc8d4f484f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:28.396250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-01638e6e-276d-4211-9f7c-8afc8d4f484f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:28.474841 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1395/2790] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:28.481605 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:28.483297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:28.558867 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7f294-006c-4c45-ad4b-20aee131f515 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:28.566669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fa0d9-1241-4a9f-b139-a0ecf329d973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:28.567873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fa0d9-1241-4a9f-b139-a0ecf329d973 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:28.567873 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fa0d9-1241-4a9f-b139-a0ecf329d973 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:28.568121 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fa0d9-1241-4a9f-b139-a0ecf329d973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:28.586627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fa0d9-1241-4a9f-b139-a0ecf329d973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:28.589741 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fa0d9-1241-4a9f-b139-a0ecf329d973 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:28.605172 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1396/2791] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:29.411988 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01638e6e-276d-4211-9f7c-8afc8d4f484f None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:29.414431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-01638e6e-276d-4211-9f7c-8afc8d4f484f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:29.429842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01638e6e-276d-4211-9f7c-8afc8d4f484f None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:29.435070 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:29.435860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:29.436241 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:29.436619 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] PATH_INFO: `/v3/projects/942cd0c15b1449edb4c9734788a3afc1` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:29.437865 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=942cd0c15b1449edb4c9734788a3afc1)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:29.439922 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:29.503094 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '942cd0c15b1449edb4c9734788a3afc1', 'request_id': 'req-0819d829-9cb4-4b57-877b-7e8ecfb88f41'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:29.525882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '942cd0c15b1449edb4c9734788a3afc1', 'request_id': 'req-0819d829-9cb4-4b57-877b-7e8ecfb88f41'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:29.534131 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '942cd0c15b1449edb4c9734788a3afc1', 'request_id': 'req-0819d829-9cb4-4b57-877b-7e8ecfb88f41'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:29.537139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:29.558109 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] The token cache is being invalidate because project 942cd0c15b1449edb4c9734788a3afc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:29.558414 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:29.559292 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:29.566769 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1396/2792] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:29 2022] DELETE /identity/v3/projects/942cd0c15b1449edb4c9734788a3afc1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:29.569460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fa0d9-1241-4a9f-b139-a0ecf329d973 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:29.582946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fa0d9-1241-4a9f-b139-a0ecf329d973 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:29.695997 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f116745-c005-4eea-a4e0-63102e4fd1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:29.696685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f116745-c005-4eea-a4e0-63102e4fd1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:29.696859 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f116745-c005-4eea-a4e0-63102e4fd1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:29.697176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f116745-c005-4eea-a4e0-63102e4fd1ed admin admin] PATH_INFO: `/v3/users/13eb11e8b3c54eb098aa8265a2309a6e` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:29.698726 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f116745-c005-4eea-a4e0-63102e4fd1ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=13eb11e8b3c54eb098aa8265a2309a6e)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:29.700952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f116745-c005-4eea-a4e0-63102e4fd1ed admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:29.784857 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:29.796533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4f116745-c005-4eea-a4e0-63102e4fd1ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13eb11e8b3c54eb098aa8265a2309a6e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:29.808205 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4f116745-c005-4eea-a4e0-63102e4fd1ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13eb11e8b3c54eb098aa8265a2309a6e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:29.809113 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4f116745-c005-4eea-a4e0-63102e4fd1ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13eb11e8b3c54eb098aa8265a2309a6e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:29.827597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4f116745-c005-4eea-a4e0-63102e4fd1ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13eb11e8b3c54eb098aa8265a2309a6e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:29.828234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-4f116745-c005-4eea-a4e0-63102e4fd1ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13eb11e8b3c54eb098aa8265a2309a6e'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:29.850499 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1397/2793] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:29 2022] DELETE /identity/v3/users/13eb11e8b3c54eb098aa8265a2309a6e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:30.223399 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:30.225104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:30.264794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819d829-9cb4-4b57-877b-7e8ecfb88f41 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:30.349015 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36d9e5-731c-4e3b-875e-a805c54e66ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:30.349674 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36d9e5-731c-4e3b-875e-a805c54e66ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:30.349958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36d9e5-731c-4e3b-875e-a805c54e66ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:30.350385 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36d9e5-731c-4e3b-875e-a805c54e66ae admin admin] PATH_INFO: `/v3/projects/1cea3999cda44b63a768019c967eac86` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:30.352317 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36d9e5-731c-4e3b-875e-a805c54e66ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cea3999cda44b63a768019c967eac86)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:30.354368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36d9e5-731c-4e3b-875e-a805c54e66ae admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:30.417407 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8f36d9e5-731c-4e3b-875e-a805c54e66ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cea3999cda44b63a768019c967eac86', 'request_id': 'req-8f36d9e5-731c-4e3b-875e-a805c54e66ae'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:30.426274 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8f36d9e5-731c-4e3b-875e-a805c54e66ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cea3999cda44b63a768019c967eac86', 'request_id': 'req-8f36d9e5-731c-4e3b-875e-a805c54e66ae'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:30.434664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8f36d9e5-731c-4e3b-875e-a805c54e66ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cea3999cda44b63a768019c967eac86', 'request_id': 'req-8f36d9e5-731c-4e3b-875e-a805c54e66ae'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:30.447356 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8f36d9e5-731c-4e3b-875e-a805c54e66ae admin admin] The token cache is being invalidate because project 1cea3999cda44b63a768019c967eac86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:30.447651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8f36d9e5-731c-4e3b-875e-a805c54e66ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:30.448499 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8f36d9e5-731c-4e3b-875e-a805c54e66ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:30.450592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1397/2794] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:30 2022] DELETE /identity/v3/projects/1cea3999cda44b63a768019c967eac86 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:30.456465 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04429ff5-d6a0-4f39-b33a-c124c3473b52 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:30.456749 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04429ff5-d6a0-4f39-b33a-c124c3473b52 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:30.457016 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04429ff5-d6a0-4f39-b33a-c124c3473b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:30.486961 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-04429ff5-d6a0-4f39-b33a-c124c3473b52 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:30.488384 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-04429ff5-d6a0-4f39-b33a-c124c3473b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:30.575924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1398/2795] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:57:30.584636 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47f855-c87f-4a7d-8cdb-c1be122bb883 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:30.585374 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47f855-c87f-4a7d-8cdb-c1be122bb883 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:30.586558 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47f855-c87f-4a7d-8cdb-c1be122bb883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:30.618534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-8b47f855-c87f-4a7d-8cdb-c1be122bb883 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:30.620378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-8b47f855-c87f-4a7d-8cdb-c1be122bb883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:30.635211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1398/2796] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:30.635955 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04429ff5-d6a0-4f39-b33a-c124c3473b52 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:30.647189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04429ff5-d6a0-4f39-b33a-c124c3473b52 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:30.655916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea331ecf-deab-4f68-8d64-02e100ae282b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:30.657196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea331ecf-deab-4f68-8d64-02e100ae282b admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:30.657196 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea331ecf-deab-4f68-8d64-02e100ae282b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:30.657432 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea331ecf-deab-4f68-8d64-02e100ae282b admin admin] PATH_INFO: `/v3/domains` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:30.657931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea331ecf-deab-4f68-8d64-02e100ae282b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:57:30.658323 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea331ecf-deab-4f68-8d64-02e100ae282b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:30.660183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea331ecf-deab-4f68-8d64-02e100ae282b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:30.680139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1399/2797] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:57:30.685396 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47f855-c87f-4a7d-8cdb-c1be122bb883 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:30.697496 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47f855-c87f-4a7d-8cdb-c1be122bb883 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:30.704687 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdf06a-f134-403b-8b09-1cc99e930e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:30.707202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdf06a-f134-403b-8b09-1cc99e930e9a admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:30.707645 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdf06a-f134-403b-8b09-1cc99e930e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:30.709497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdf06a-f134-403b-8b09-1cc99e930e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:30.710332 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdf06a-f134-403b-8b09-1cc99e930e9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:30.713155 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdf06a-f134-403b-8b09-1cc99e930e9a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:30.765696 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1399/2798] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:30 2022] POST /identity/v3/projects => generated 397 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:57:30.770280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea331ecf-deab-4f68-8d64-02e100ae282b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:30.782160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea331ecf-deab-4f68-8d64-02e100ae282b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:30.878305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316045a2-9784-4ea1-9435-13b11be4d423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:30.879223 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316045a2-9784-4ea1-9435-13b11be4d423 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:30.879662 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316045a2-9784-4ea1-9435-13b11be4d423 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:30.880087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316045a2-9784-4ea1-9435-13b11be4d423 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:30.880958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316045a2-9784-4ea1-9435-13b11be4d423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:30.883863 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316045a2-9784-4ea1-9435-13b11be4d423 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:30.919651 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1400/2799] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:30 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:57:30.924526 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdf06a-f134-403b-8b09-1cc99e930e9a admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:30.940175 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdf06a-f134-403b-8b09-1cc99e930e9a admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:30.946749 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f325f-b977-4e23-b0a9-d16389a92dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:30.947497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f325f-b977-4e23-b0a9-d16389a92dcc admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:30.947847 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f325f-b977-4e23-b0a9-d16389a92dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:30.948202 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f325f-b977-4e23-b0a9-d16389a92dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:30.948937 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f325f-b977-4e23-b0a9-d16389a92dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:30.950705 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f325f-b977-4e23-b0a9-d16389a92dcc admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:30.965317 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1400/2800] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:57:30.969601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316045a2-9784-4ea1-9435-13b11be4d423 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:30.985004 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316045a2-9784-4ea1-9435-13b11be4d423 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:30.996852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d04b4-220b-4e0c-8849-7794b8173fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:30.996852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d04b4-220b-4e0c-8849-7794b8173fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:30.996852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d04b4-220b-4e0c-8849-7794b8173fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:30.996852 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d04b4-220b-4e0c-8849-7794b8173fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:30.998069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d04b4-220b-4e0c-8849-7794b8173fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:30.999855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d04b4-220b-4e0c-8849-7794b8173fb4 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:31.014533 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1401/2801] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:31.018066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f325f-b977-4e23-b0a9-d16389a92dcc admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:31.037611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f325f-b977-4e23-b0a9-d16389a92dcc admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:31.047211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08810622-4648-444e-9555-578bd490059a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:31.048600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08810622-4648-444e-9555-578bd490059a admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:31.049227 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08810622-4648-444e-9555-578bd490059a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:31.049615 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08810622-4648-444e-9555-578bd490059a admin admin] PATH_INFO: `/v3/projects/3f6ff4d090e64c28a6b80a0dddd2d880/users/9a833d6668494db9b11a80f7809fd69a/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:31.088132 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08810622-4648-444e-9555-578bd490059a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6ff4d090e64c28a6b80a0dddd2d880, user_id=9a833d6668494db9b11a80f7809fd69a, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:31.092821 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08810622-4648-444e-9555-578bd490059a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:31.123221 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1401/2802] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:31 2022] PUT /identity/v3/projects/3f6ff4d090e64c28a6b80a0dddd2d880/users/9a833d6668494db9b11a80f7809fd69a/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:31.138137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd554a5b-9c39-4d78-93df-e8e61859f7b2 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:31.138137 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd554a5b-9c39-4d78-93df-e8e61859f7b2 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:31.138449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd554a5b-9c39-4d78-93df-e8e61859f7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:31.184231 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-cd554a5b-9c39-4d78-93df-e8e61859f7b2 None None] MFA Rules not processed for user `9a833d6668494db9b11a80f7809fd69a`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:31.187319 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-cd554a5b-9c39-4d78-93df-e8e61859f7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:31.228469 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:31.314773 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1402/2803] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:31 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:31.320004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08810622-4648-444e-9555-578bd490059a admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:31.343842 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:31.433465 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08810622-4648-444e-9555-578bd490059a admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:31.444941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3029c26-bf9b-48ba-a85c-857a87c5ebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:31.445654 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3029c26-bf9b-48ba-a85c-857a87c5ebe5 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:31.445941 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3029c26-bf9b-48ba-a85c-857a87c5ebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:31.446217 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3029c26-bf9b-48ba-a85c-857a87c5ebe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:31.446877 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3029c26-bf9b-48ba-a85c-857a87c5ebe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:31.448887 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3029c26-bf9b-48ba-a85c-857a87c5ebe5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:31.505024 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1402/2804] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:31 2022] POST /identity/v3/projects => generated 399 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:57:31.509496 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd554a5b-9c39-4d78-93df-e8e61859f7b2 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:31.521652 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd554a5b-9c39-4d78-93df-e8e61859f7b2 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:31.604156 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06622b-a70e-4eb9-803f-21d397aeac02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:31.605026 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06622b-a70e-4eb9-803f-21d397aeac02 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:31.605848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06622b-a70e-4eb9-803f-21d397aeac02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:31.605848 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06622b-a70e-4eb9-803f-21d397aeac02 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:31.606338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06622b-a70e-4eb9-803f-21d397aeac02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:31.608436 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06622b-a70e-4eb9-803f-21d397aeac02 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:31.641718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1403/2805] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:31 2022] POST /identity/v3/users => generated 306 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:31.642290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3029c26-bf9b-48ba-a85c-857a87c5ebe5 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:31.656719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3029c26-bf9b-48ba-a85c-857a87c5ebe5 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:31.664334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124c9be-f4a3-4ef6-82ee-74333fdef4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:31.665317 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124c9be-f4a3-4ef6-82ee-74333fdef4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:31.665951 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124c9be-f4a3-4ef6-82ee-74333fdef4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:31.666457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124c9be-f4a3-4ef6-82ee-74333fdef4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:31.667414 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124c9be-f4a3-4ef6-82ee-74333fdef4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:31.669457 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124c9be-f4a3-4ef6-82ee-74333fdef4a4 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:31.693692 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1403/2806] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:31.698911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06622b-a70e-4eb9-803f-21d397aeac02 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:31.717995 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06622b-a70e-4eb9-803f-21d397aeac02 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:31.724049 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811328a5-7993-4321-870e-abdb13a2e773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:31.724750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811328a5-7993-4321-870e-abdb13a2e773 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:31.725029 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811328a5-7993-4321-870e-abdb13a2e773 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:31.725356 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811328a5-7993-4321-870e-abdb13a2e773 admin admin] PATH_INFO: `/v3/projects/7e7b120e451d473c97696bc7ccb39d34/users/35f8557d632d4f31998de073098e8f5d/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:31.742908 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811328a5-7993-4321-870e-abdb13a2e773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e7b120e451d473c97696bc7ccb39d34, user_id=35f8557d632d4f31998de073098e8f5d, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:31.746174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811328a5-7993-4321-870e-abdb13a2e773 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:31.786173 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1404/2807] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:31 2022] PUT /identity/v3/projects/7e7b120e451d473c97696bc7ccb39d34/users/35f8557d632d4f31998de073098e8f5d/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:31.790997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124c9be-f4a3-4ef6-82ee-74333fdef4a4 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:31.809045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124c9be-f4a3-4ef6-82ee-74333fdef4a4 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:31.898118 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4060c-92a1-4c85-9896-a42bc4537791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:31.899303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4060c-92a1-4c85-9896-a42bc4537791 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:31.899690 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4060c-92a1-4c85-9896-a42bc4537791 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:31.900079 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4060c-92a1-4c85-9896-a42bc4537791 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:31.900830 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4060c-92a1-4c85-9896-a42bc4537791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:31.903150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4060c-92a1-4c85-9896-a42bc4537791 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:31.915527 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1404/2808] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:31.919493 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811328a5-7993-4321-870e-abdb13a2e773 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:31.933443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811328a5-7993-4321-870e-abdb13a2e773 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:31.938858 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b2f9a-18d1-4e90-8fc1-6b45fea6c0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:31.939585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b2f9a-18d1-4e90-8fc1-6b45fea6c0fd admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:31.939865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b2f9a-18d1-4e90-8fc1-6b45fea6c0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:31.940138 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b2f9a-18d1-4e90-8fc1-6b45fea6c0fd admin admin] PATH_INFO: `/v3/projects/7e7b120e451d473c97696bc7ccb39d34/users/35f8557d632d4f31998de073098e8f5d/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:31.942280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b2f9a-18d1-4e90-8fc1-6b45fea6c0fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e7b120e451d473c97696bc7ccb39d34, user_id=35f8557d632d4f31998de073098e8f5d, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:31.946073 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b2f9a-18d1-4e90-8fc1-6b45fea6c0fd admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:31.958160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1405/2809] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:31 2022] PUT /identity/v3/projects/7e7b120e451d473c97696bc7ccb39d34/users/35f8557d632d4f31998de073098e8f5d/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:31.971184 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3da0f-e49f-4532-bf3e-1154d782e08e None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:31.971511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3da0f-e49f-4532-bf3e-1154d782e08e None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:31.971826 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3da0f-e49f-4532-bf3e-1154d782e08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:32.007799 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-62e3da0f-e49f-4532-bf3e-1154d782e08e None None] MFA Rules not processed for user `35f8557d632d4f31998de073098e8f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:32.009050 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-62e3da0f-e49f-4532-bf3e-1154d782e08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:32.135134 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1405/2810] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:31 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:32.147488 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b2f9a-18d1-4e90-8fc1-6b45fea6c0fd admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:32.149598 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-815b2f9a-18d1-4e90-8fc1-6b45fea6c0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:32.244667 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b2f9a-18d1-4e90-8fc1-6b45fea6c0fd admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:32.251917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370be8a-e398-4161-b63e-194044fd38c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:32.253388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370be8a-e398-4161-b63e-194044fd38c6 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:32.254101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370be8a-e398-4161-b63e-194044fd38c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:32.254811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370be8a-e398-4161-b63e-194044fd38c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:32.276314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370be8a-e398-4161-b63e-194044fd38c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:32.277941 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370be8a-e398-4161-b63e-194044fd38c6 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:32.300157 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1406/2811] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:32 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:32.361794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee7259-aaef-466d-b6f3-d086ab29c3f3 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:32.362129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee7259-aaef-466d-b6f3-d086ab29c3f3 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:32.362337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee7259-aaef-466d-b6f3-d086ab29c3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:32.396246 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:32.406180 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-17ee7259-aaef-466d-b6f3-d086ab29c3f3 None None] MFA Rules not processed for user `f4003700359449afb83a7352ff2b326a`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:32.407623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-17ee7259-aaef-466d-b6f3-d086ab29c3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:32.412682 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370be8a-e398-4161-b63e-194044fd38c6 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:32.435921 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370be8a-e398-4161-b63e-194044fd38c6 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:32.441261 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdd0c4-5187-46b8-b226-5abcaafb7e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:32.442639 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdd0c4-5187-46b8-b226-5abcaafb7e80 service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:32.443179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdd0c4-5187-46b8-b226-5abcaafb7e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:32.443669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdd0c4-5187-46b8-b226-5abcaafb7e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:32.459215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:32.529130 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1406/2812] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:32.531736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdd0c4-5187-46b8-b226-5abcaafb7e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:32.533449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdd0c4-5187-46b8-b226-5abcaafb7e80 service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:32.533725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee7259-aaef-466d-b6f3-d086ab29c3f3 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:32.535311 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-17ee7259-aaef-466d-b6f3-d086ab29c3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:32.549720 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1407/2813] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:57:32.635546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee7259-aaef-466d-b6f3-d086ab29c3f3 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:32.643817 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1be4a-c8d2-477a-8307-20e45ef673e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:32.644793 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1be4a-c8d2-477a-8307-20e45ef673e2 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:32.645146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1be4a-c8d2-477a-8307-20e45ef673e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:32.645740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1be4a-c8d2-477a-8307-20e45ef673e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:32.646497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1be4a-c8d2-477a-8307-20e45ef673e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:32.650087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1be4a-c8d2-477a-8307-20e45ef673e2 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:32.681448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1407/2814] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:32 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:57:32.685510 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdd0c4-5187-46b8-b226-5abcaafb7e80 service cinder] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:32.698373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdd0c4-5187-46b8-b226-5abcaafb7e80 service cinder] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:32.775340 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4d6f8-12a7-4c77-a80f-c6e1a05248a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:32.776352 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4d6f8-12a7-4c77-a80f-c6e1a05248a8 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:32.776916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4d6f8-12a7-4c77-a80f-c6e1a05248a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:32.777401 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4d6f8-12a7-4c77-a80f-c6e1a05248a8 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:32.778374 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4d6f8-12a7-4c77-a80f-c6e1a05248a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:32.780351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4d6f8-12a7-4c77-a80f-c6e1a05248a8 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:32.806089 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1408/2815] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:32 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:32.810497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1be4a-c8d2-477a-8307-20e45ef673e2 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:32.824464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1be4a-c8d2-477a-8307-20e45ef673e2 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:32.831327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34770262-7b23-4ada-a917-f2c687b95205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:32.831945 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34770262-7b23-4ada-a917-f2c687b95205 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:32.832316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34770262-7b23-4ada-a917-f2c687b95205 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:32.832589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34770262-7b23-4ada-a917-f2c687b95205 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:32.833265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34770262-7b23-4ada-a917-f2c687b95205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:32.835476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34770262-7b23-4ada-a917-f2c687b95205 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:32.847076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1408/2816] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:32.851754 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4d6f8-12a7-4c77-a80f-c6e1a05248a8 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:32.863492 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4d6f8-12a7-4c77-a80f-c6e1a05248a8 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:32.869676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818d324-4049-453d-bbf4-03d6dfaef787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:32.870350 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818d324-4049-453d-bbf4-03d6dfaef787 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:32.870626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818d324-4049-453d-bbf4-03d6dfaef787 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:32.870903 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818d324-4049-453d-bbf4-03d6dfaef787 admin admin] PATH_INFO: `/v3/projects/ef85975f274544618475ececc91cda31/users/51abe0e78eef423ca0bd6b80533311bb/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:32.893209 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818d324-4049-453d-bbf4-03d6dfaef787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef85975f274544618475ececc91cda31, user_id=51abe0e78eef423ca0bd6b80533311bb, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:32.896306 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818d324-4049-453d-bbf4-03d6dfaef787 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:32.917432 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1409/2817] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:32 2022] PUT /identity/v3/projects/ef85975f274544618475ececc91cda31/users/51abe0e78eef423ca0bd6b80533311bb/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:32.924706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34770262-7b23-4ada-a917-f2c687b95205 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:32.936564 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34770262-7b23-4ada-a917-f2c687b95205 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:33.004115 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c67f5-9d2c-4b2e-8be7-c2afb9a5a81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:33.004756 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c67f5-9d2c-4b2e-8be7-c2afb9a5a81d admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:33.005039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c67f5-9d2c-4b2e-8be7-c2afb9a5a81d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:33.005390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c67f5-9d2c-4b2e-8be7-c2afb9a5a81d admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:33.006253 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c67f5-9d2c-4b2e-8be7-c2afb9a5a81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:33.008859 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c67f5-9d2c-4b2e-8be7-c2afb9a5a81d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:33.019025 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1409/2818] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:33.024012 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818d324-4049-453d-bbf4-03d6dfaef787 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:33.034327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818d324-4049-453d-bbf4-03d6dfaef787 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:33.039078 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4f90a-8f9a-4113-8e8e-2057b911d9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:33.039750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4f90a-8f9a-4113-8e8e-2057b911d9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:33.040028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4f90a-8f9a-4113-8e8e-2057b911d9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:33.040299 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4f90a-8f9a-4113-8e8e-2057b911d9ae admin admin] PATH_INFO: `/v3/projects/ef85975f274544618475ececc91cda31/users/51abe0e78eef423ca0bd6b80533311bb/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:33.042212 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4f90a-8f9a-4113-8e8e-2057b911d9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef85975f274544618475ececc91cda31, user_id=51abe0e78eef423ca0bd6b80533311bb, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:33.044911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4f90a-8f9a-4113-8e8e-2057b911d9ae admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:33.055310 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1410/2819] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:33 2022] PUT /identity/v3/projects/ef85975f274544618475ececc91cda31/users/51abe0e78eef423ca0bd6b80533311bb/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:57:37.356431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c67f5-9d2c-4b2e-8be7-c2afb9a5a81d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:37.383201 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c67f5-9d2c-4b2e-8be7-c2afb9a5a81d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:37.413623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:37.548661 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:37.551620 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:37.552732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:37.553741 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] PATH_INFO: `/v3/users/9a833d6668494db9b11a80f7809fd69a` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:37.557618 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a833d6668494db9b11a80f7809fd69a)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:37.564781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:37.723874 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a833d6668494db9b11a80f7809fd69a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:37.759933 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a833d6668494db9b11a80f7809fd69a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:37.760721 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a833d6668494db9b11a80f7809fd69a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:37.761360 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a833d6668494db9b11a80f7809fd69a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:37.773023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a833d6668494db9b11a80f7809fd69a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:37.793353 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1410/2820] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:37 2022] DELETE /identity/v3/users/9a833d6668494db9b11a80f7809fd69a => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:37.796000 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271185e-a207-4f7c-a00a-b7af381ba104 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:37.797151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271185e-a207-4f7c-a00a-b7af381ba104 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:37.797628 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271185e-a207-4f7c-a00a-b7af381ba104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:37.848602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-6271185e-a207-4f7c-a00a-b7af381ba104 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:37.852311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6271185e-a207-4f7c-a00a-b7af381ba104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:38.007178 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1411/2821] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:38.018090 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:38.019963 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:38.166667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bcdc7-860f-4674-a455-84a6a28bd565 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:38.173337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8f2a8-f403-4c1f-b6f4-98a1b62b617a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:38.174302 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8f2a8-f403-4c1f-b6f4-98a1b62b617a service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:38.174958 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8f2a8-f403-4c1f-b6f4-98a1b62b617a service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:38.175446 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8f2a8-f403-4c1f-b6f4-98a1b62b617a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:38.191709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8f2a8-f403-4c1f-b6f4-98a1b62b617a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:38.195579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8f2a8-f403-4c1f-b6f4-98a1b62b617a service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:38.211035 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1411/2822] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:38.673598 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271185e-a207-4f7c-a00a-b7af381ba104 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:38.674975 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-6271185e-a207-4f7c-a00a-b7af381ba104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:38.686597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:38.693281 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271185e-a207-4f7c-a00a-b7af381ba104 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:38.699973 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:38.703757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:38.704151 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:38.704946 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] PATH_INFO: `/v3/projects/3f6ff4d090e64c28a6b80a0dddd2d880` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:38.709140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f6ff4d090e64c28a6b80a0dddd2d880)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:38.713203 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:38.845336 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f6ff4d090e64c28a6b80a0dddd2d880', 'request_id': 'req-ebf39461-4afb-49a7-bf1c-96b4e06554d1'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:38.859534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f6ff4d090e64c28a6b80a0dddd2d880', 'request_id': 'req-ebf39461-4afb-49a7-bf1c-96b4e06554d1'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:38.866496 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f6ff4d090e64c28a6b80a0dddd2d880', 'request_id': 'req-ebf39461-4afb-49a7-bf1c-96b4e06554d1'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:38.886238 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d3e40-0deb-43d2-b241-35ffb177c6bb None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:38.886648 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d3e40-0deb-43d2-b241-35ffb177c6bb None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:38.886761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d3e40-0deb-43d2-b241-35ffb177c6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:38.889638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] The token cache is being invalidate because project 3f6ff4d090e64c28a6b80a0dddd2d880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:38.889998 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:38.890837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:38.892636 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1412/2823] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:38 2022] DELETE /identity/v3/projects/3f6ff4d090e64c28a6b80a0dddd2d880 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:38.896864 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:38.899625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:38.942077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-bb2d3e40-0deb-43d2-b241-35ffb177c6bb None None] MFA Rules not processed for user `51abe0e78eef423ca0bd6b80533311bb`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:38.943797 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-bb2d3e40-0deb-43d2-b241-35ffb177c6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:38.979451 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:38.984297 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf39461-4afb-49a7-bf1c-96b4e06554d1 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:38.989500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:38.990293 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:38.990606 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:38.991817 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] PATH_INFO: `/v3/users/35f8557d632d4f31998de073098e8f5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:38.993392 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f8557d632d4f31998de073098e8f5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:38.995541 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:39.042681 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1412/2824] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:39.058536 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d3e40-0deb-43d2-b241-35ffb177c6bb None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:39.059985 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-bb2d3e40-0deb-43d2-b241-35ffb177c6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:39.066401 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f8557d632d4f31998de073098e8f5d'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:39.078564 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f8557d632d4f31998de073098e8f5d'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:39.079938 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f8557d632d4f31998de073098e8f5d'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:39.105676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f8557d632d4f31998de073098e8f5d'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:39.107320 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f8557d632d4f31998de073098e8f5d'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:39.126153 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1413/2825] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:38 2022] DELETE /identity/v3/users/35f8557d632d4f31998de073098e8f5d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:57:39.214352 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d3e40-0deb-43d2-b241-35ffb177c6bb None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:39.303938 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8d54f-6898-4716-9464-ec0115682df6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:39.304860 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8d54f-6898-4716-9464-ec0115682df6 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:39.305213 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8d54f-6898-4716-9464-ec0115682df6 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:39.305562 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8d54f-6898-4716-9464-ec0115682df6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:39.307943 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-8ee8d54f-6898-4716-9464-ec0115682df6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:39.347716 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=100 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:39.389104 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8d54f-6898-4716-9464-ec0115682df6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:39.392324 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8d54f-6898-4716-9464-ec0115682df6 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:39.408614 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1413/2826] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:39.808432 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:39.811263 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:39.847314 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:39.963144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2220480-6c21-42ec-9994-b30efe7ffe5e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:39.968425 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b97dca-d193-499b-a6da-3f0b67d8a8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:39.969170 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b97dca-d193-499b-a6da-3f0b67d8a8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:39.969522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b97dca-d193-499b-a6da-3f0b67d8a8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:39.969867 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b97dca-d193-499b-a6da-3f0b67d8a8fa admin admin] PATH_INFO: `/v3/projects/7e7b120e451d473c97696bc7ccb39d34` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:39.971139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b97dca-d193-499b-a6da-3f0b67d8a8fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e7b120e451d473c97696bc7ccb39d34)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:39.973215 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b97dca-d193-499b-a6da-3f0b67d8a8fa admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:40.039931 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-50b97dca-d193-499b-a6da-3f0b67d8a8fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e7b120e451d473c97696bc7ccb39d34', 'request_id': 'req-50b97dca-d193-499b-a6da-3f0b67d8a8fa'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:40.047627 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-50b97dca-d193-499b-a6da-3f0b67d8a8fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e7b120e451d473c97696bc7ccb39d34', 'request_id': 'req-50b97dca-d193-499b-a6da-3f0b67d8a8fa'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:40.054952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-50b97dca-d193-499b-a6da-3f0b67d8a8fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e7b120e451d473c97696bc7ccb39d34', 'request_id': 'req-50b97dca-d193-499b-a6da-3f0b67d8a8fa'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:40.074588 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-50b97dca-d193-499b-a6da-3f0b67d8a8fa admin admin] The token cache is being invalidate because project 7e7b120e451d473c97696bc7ccb39d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:40.075303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-50b97dca-d193-499b-a6da-3f0b67d8a8fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:40.076725 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-50b97dca-d193-499b-a6da-3f0b67d8a8fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:40.079552 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1414/2827] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:39 2022] DELETE /identity/v3/projects/7e7b120e451d473c97696bc7ccb39d34 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:40.085236 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73449426-dbc7-470c-859d-51b43277cc27 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:40.085950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73449426-dbc7-470c-859d-51b43277cc27 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:40.085950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73449426-dbc7-470c-859d-51b43277cc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:40.120500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-73449426-dbc7-470c-859d-51b43277cc27 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:40.121855 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-73449426-dbc7-470c-859d-51b43277cc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:40.216853 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1414/2828] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:40.224618 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7894c-b267-4381-91bf-e6f925711f47 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:40.225119 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7894c-b267-4381-91bf-e6f925711f47 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:40.225601 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7894c-b267-4381-91bf-e6f925711f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:40.263751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-f0f7894c-b267-4381-91bf-e6f925711f47 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:40.265942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f0f7894c-b267-4381-91bf-e6f925711f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:40.282549 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1415/2829] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:40.283380 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73449426-dbc7-470c-859d-51b43277cc27 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:40.302959 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73449426-dbc7-470c-859d-51b43277cc27 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:40.309387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155c2f3-132b-4b85-b92a-5d30113c27f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:40.310019 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155c2f3-132b-4b85-b92a-5d30113c27f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:40.310303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155c2f3-132b-4b85-b92a-5d30113c27f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:40.310579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155c2f3-132b-4b85-b92a-5d30113c27f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:40.311326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155c2f3-132b-4b85-b92a-5d30113c27f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:57:40.311626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155c2f3-132b-4b85-b92a-5d30113c27f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:40.313420 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155c2f3-132b-4b85-b92a-5d30113c27f5 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:40.333589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1415/2830] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:57:40.339017 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7894c-b267-4381-91bf-e6f925711f47 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:40.350175 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7894c-b267-4381-91bf-e6f925711f47 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:40.373086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51b261-0d1f-46b1-b8ad-b83c92a4fdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:40.373739 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51b261-0d1f-46b1-b8ad-b83c92a4fdec admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:40.374015 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51b261-0d1f-46b1-b8ad-b83c92a4fdec admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:40.374290 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51b261-0d1f-46b1-b8ad-b83c92a4fdec admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:40.374935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51b261-0d1f-46b1-b8ad-b83c92a4fdec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:40.376924 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51b261-0d1f-46b1-b8ad-b83c92a4fdec admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:40.415981 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1416/2831] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:40 2022] POST /identity/v3/projects => generated 389 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:40.417522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155c2f3-132b-4b85-b92a-5d30113c27f5 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:40.434424 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155c2f3-132b-4b85-b92a-5d30113c27f5 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:40.520646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33850dfd-a285-41e1-b562-df0032ab727f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:40.520646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33850dfd-a285-41e1-b562-df0032ab727f admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:40.520646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33850dfd-a285-41e1-b562-df0032ab727f admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:40.520646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33850dfd-a285-41e1-b562-df0032ab727f admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:40.521026 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33850dfd-a285-41e1-b562-df0032ab727f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:40.522646 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33850dfd-a285-41e1-b562-df0032ab727f admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:40.550408 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1416/2832] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:40 2022] POST /identity/v3/users => generated 295 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:40.555439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51b261-0d1f-46b1-b8ad-b83c92a4fdec admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:40.568803 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51b261-0d1f-46b1-b8ad-b83c92a4fdec admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:40.575190 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7181c-909c-46db-8650-3bcedfcb705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:40.575950 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7181c-909c-46db-8650-3bcedfcb705e admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:40.576727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7181c-909c-46db-8650-3bcedfcb705e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:40.576727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7181c-909c-46db-8650-3bcedfcb705e admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:40.577117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7181c-909c-46db-8650-3bcedfcb705e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:40.579098 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7181c-909c-46db-8650-3bcedfcb705e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:40.594108 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1417/2833] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:40.597961 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33850dfd-a285-41e1-b562-df0032ab727f admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:40.610532 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33850dfd-a285-41e1-b562-df0032ab727f admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:40.616898 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74123c8-e6a9-4ccc-8cbb-b0371c4ea8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:40.617742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74123c8-e6a9-4ccc-8cbb-b0371c4ea8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:40.618390 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74123c8-e6a9-4ccc-8cbb-b0371c4ea8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:40.618782 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74123c8-e6a9-4ccc-8cbb-b0371c4ea8b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:40.619606 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74123c8-e6a9-4ccc-8cbb-b0371c4ea8b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:40.621426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74123c8-e6a9-4ccc-8cbb-b0371c4ea8b8 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:40.633433 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1417/2834] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:40.640337 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7181c-909c-46db-8650-3bcedfcb705e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:40.652250 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7181c-909c-46db-8650-3bcedfcb705e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:40.657679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09ad38-1542-44ed-a7f0-35fa8926c107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:40.658630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09ad38-1542-44ed-a7f0-35fa8926c107 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:40.659497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09ad38-1542-44ed-a7f0-35fa8926c107 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:40.659497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09ad38-1542-44ed-a7f0-35fa8926c107 admin admin] PATH_INFO: `/v3/projects/a8296806f80f4ef6b9b048b43da68af7/users/ee29d4ca248d487da1e03a1b2d710858/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:40.683050 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09ad38-1542-44ed-a7f0-35fa8926c107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8296806f80f4ef6b9b048b43da68af7, user_id=ee29d4ca248d487da1e03a1b2d710858, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:40.685939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09ad38-1542-44ed-a7f0-35fa8926c107 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:40.702498 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1418/2835] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:40 2022] PUT /identity/v3/projects/a8296806f80f4ef6b9b048b43da68af7/users/ee29d4ca248d487da1e03a1b2d710858/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:40.716216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910ff41-0271-476a-8f5e-d570c8540bdd None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:40.716216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910ff41-0271-476a-8f5e-d570c8540bdd None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:40.720327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910ff41-0271-476a-8f5e-d570c8540bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:40.759178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-b910ff41-0271-476a-8f5e-d570c8540bdd None None] MFA Rules not processed for user `ee29d4ca248d487da1e03a1b2d710858`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:40.763188 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-b910ff41-0271-476a-8f5e-d570c8540bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:40.798245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:40.858184 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c9d55-3469-4b81-b8d8-7ffcf3f45bfe None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:40.858879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c9d55-3469-4b81-b8d8-7ffcf3f45bfe None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:40.858879 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c9d55-3469-4b81-b8d8-7ffcf3f45bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:40.861002 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1418/2836] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:40.866366 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910ff41-0271-476a-8f5e-d570c8540bdd None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:40.880743 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-671c9d55-3469-4b81-b8d8-7ffcf3f45bfe None None] MFA Rules not processed for user `f4003700359449afb83a7352ff2b326a`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:40.882037 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-671c9d55-3469-4b81-b8d8-7ffcf3f45bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:40.926069 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:40.950728 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910ff41-0271-476a-8f5e-d570c8540bdd None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:40.955579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4405c9-d8c9-4086-92a7-fb3bd0030271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:40.956195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4405c9-d8c9-4086-92a7-fb3bd0030271 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:40.956486 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4405c9-d8c9-4086-92a7-fb3bd0030271 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:40.956764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4405c9-d8c9-4086-92a7-fb3bd0030271 admin admin] PATH_INFO: `/v3/projects` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:40.957413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4405c9-d8c9-4086-92a7-fb3bd0030271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:40.959400 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4405c9-d8c9-4086-92a7-fb3bd0030271 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:40.963251 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1419/2837] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:57:40.995030 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1419/2838] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:40 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:57:40.999754 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c9d55-3469-4b81-b8d8-7ffcf3f45bfe None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.026139 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c9d55-3469-4b81-b8d8-7ffcf3f45bfe None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.117682 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66ba5c-5ac6-4ea1-88a3-2da482a73813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.119008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66ba5c-5ac6-4ea1-88a3-2da482a73813 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.119535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66ba5c-5ac6-4ea1-88a3-2da482a73813 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.120010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66ba5c-5ac6-4ea1-88a3-2da482a73813 admin admin] PATH_INFO: `/v3/users` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.121329 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66ba5c-5ac6-4ea1-88a3-2da482a73813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.124762 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66ba5c-5ac6-4ea1-88a3-2da482a73813 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.156958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1420/2839] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:40 2022] POST /identity/v3/users => generated 300 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:41.160820 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4405c9-d8c9-4086-92a7-fb3bd0030271 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.174597 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4405c9-d8c9-4086-92a7-fb3bd0030271 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.182468 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d42de-01e2-4515-8409-f3bd9af72fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.183375 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d42de-01e2-4515-8409-f3bd9af72fcd admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.183765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d42de-01e2-4515-8409-f3bd9af72fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.183942 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d42de-01e2-4515-8409-f3bd9af72fcd admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.184664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d42de-01e2-4515-8409-f3bd9af72fcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.186133 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d42de-01e2-4515-8409-f3bd9af72fcd admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.197447 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1420/2840] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:41.201778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66ba5c-5ac6-4ea1-88a3-2da482a73813 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.213599 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66ba5c-5ac6-4ea1-88a3-2da482a73813 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.223233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6c7c2-df1f-49a7-ab22-f0adde011ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.224102 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6c7c2-df1f-49a7-ab22-f0adde011ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.224736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6c7c2-df1f-49a7-ab22-f0adde011ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.224736 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6c7c2-df1f-49a7-ab22-f0adde011ac6 admin admin] PATH_INFO: `/v3/projects/c9fb5a0cd7b14b868184fc4f685dc270/users/588af476986a4ae2add9cb1408df6c0e/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.245378 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6c7c2-df1f-49a7-ab22-f0adde011ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9fb5a0cd7b14b868184fc4f685dc270, user_id=588af476986a4ae2add9cb1408df6c0e, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.248760 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6c7c2-df1f-49a7-ab22-f0adde011ac6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.265978 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1421/2841] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:41 2022] PUT /identity/v3/projects/c9fb5a0cd7b14b868184fc4f685dc270/users/588af476986a4ae2add9cb1408df6c0e/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:41.269783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d42de-01e2-4515-8409-f3bd9af72fcd admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.281475 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d42de-01e2-4515-8409-f3bd9af72fcd admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.362757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e506916-11c1-46a3-8f4b-5ce533bac3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.363416 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e506916-11c1-46a3-8f4b-5ce533bac3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.363698 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e506916-11c1-46a3-8f4b-5ce533bac3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.363977 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e506916-11c1-46a3-8f4b-5ce533bac3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.364630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e506916-11c1-46a3-8f4b-5ce533bac3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.379153 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e506916-11c1-46a3-8f4b-5ce533bac3e7 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.392276 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1421/2842] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:41.397596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6c7c2-df1f-49a7-ab22-f0adde011ac6 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.414929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6c7c2-df1f-49a7-ab22-f0adde011ac6 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.421315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55813ca-7d0d-4375-9197-ebedfd9cf613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.422430 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55813ca-7d0d-4375-9197-ebedfd9cf613 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.422830 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55813ca-7d0d-4375-9197-ebedfd9cf613 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.423274 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55813ca-7d0d-4375-9197-ebedfd9cf613 admin admin] PATH_INFO: `/v3/projects/c9fb5a0cd7b14b868184fc4f685dc270/users/588af476986a4ae2add9cb1408df6c0e/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.426260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55813ca-7d0d-4375-9197-ebedfd9cf613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9fb5a0cd7b14b868184fc4f685dc270, user_id=588af476986a4ae2add9cb1408df6c0e, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.430374 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55813ca-7d0d-4375-9197-ebedfd9cf613 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.445718 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1422/2843] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:41 2022] PUT /identity/v3/projects/c9fb5a0cd7b14b868184fc4f685dc270/users/588af476986a4ae2add9cb1408df6c0e/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:57:41.523041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ed646-e479-4fe5-ab4e-df7559b811e9 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.523368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ed646-e479-4fe5-ab4e-df7559b811e9 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.523638 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ed646-e479-4fe5-ab4e-df7559b811e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.558259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-530ed646-e479-4fe5-ab4e-df7559b811e9 None None] MFA Rules not processed for user `588af476986a4ae2add9cb1408df6c0e`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:41.559772 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-530ed646-e479-4fe5-ab4e-df7559b811e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:41.625151 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:41.688113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1422/2844] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:41.700086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55813ca-7d0d-4375-9197-ebedfd9cf613 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.702045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-d55813ca-7d0d-4375-9197-ebedfd9cf613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:41.776755 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55813ca-7d0d-4375-9197-ebedfd9cf613 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.783878 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823cddd-b1d0-4a09-abd4-2aeb9d7b578e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.784809 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823cddd-b1d0-4a09-abd4-2aeb9d7b578e service cinder] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.785106 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823cddd-b1d0-4a09-abd4-2aeb9d7b578e service cinder] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.785380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823cddd-b1d0-4a09-abd4-2aeb9d7b578e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.798467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823cddd-b1d0-4a09-abd4-2aeb9d7b578e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.799991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823cddd-b1d0-4a09-abd4-2aeb9d7b578e service cinder] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.812239 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1423/2845] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:41.982560 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ed646-e479-4fe5-ab4e-df7559b811e9 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:42.097337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ed646-e479-4fe5-ab4e-df7559b811e9 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:42.104264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:42.105129 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:42.105400 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:42.105765 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] PATH_INFO: `/v3/users/ee29d4ca248d487da1e03a1b2d710858` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:42.107195 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee29d4ca248d487da1e03a1b2d710858)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:42.109404 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:42.175923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee29d4ca248d487da1e03a1b2d710858'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:42.198358 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee29d4ca248d487da1e03a1b2d710858'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:42.199783 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee29d4ca248d487da1e03a1b2d710858'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:42.200764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee29d4ca248d487da1e03a1b2d710858'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:42.212661 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee29d4ca248d487da1e03a1b2d710858'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:42.239552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1423/2846] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:41 2022] DELETE /identity/v3/users/ee29d4ca248d487da1e03a1b2d710858 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:57:42.246417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5e961-619b-4cf7-90ba-feb0122f173f None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:42.248535 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5e961-619b-4cf7-90ba-feb0122f173f None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:42.249240 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5e961-619b-4cf7-90ba-feb0122f173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:42.277159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-92a5e961-619b-4cf7-90ba-feb0122f173f None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:42.279086 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-92a5e961-619b-4cf7-90ba-feb0122f173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:42.329118 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:42.398530 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1424/2847] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:42.409699 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:42.411308 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:42.490162 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:42.525061 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ebad6-2dcb-43d4-b657-0c57fc6e24d8 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:42.531461 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d45cb-3805-4fe2-a7b2-c301a9660a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:42.532431 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d45cb-3805-4fe2-a7b2-c301a9660a2e service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:42.532881 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d45cb-3805-4fe2-a7b2-c301a9660a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:42.533341 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d45cb-3805-4fe2-a7b2-c301a9660a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:42.545284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d45cb-3805-4fe2-a7b2-c301a9660a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:42.546732 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d45cb-3805-4fe2-a7b2-c301a9660a2e service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:42.558630 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1424/2848] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:42.930133 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5e961-619b-4cf7-90ba-feb0122f173f None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:42.932647 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-92a5e961-619b-4cf7-90ba-feb0122f173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:42.952075 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5e961-619b-4cf7-90ba-feb0122f173f None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:42.957669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:42.960280 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:42.960695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:42.961257 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] PATH_INFO: `/v3/projects/a8296806f80f4ef6b9b048b43da68af7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:42.964402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8296806f80f4ef6b9b048b43da68af7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:42.967791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:43.021192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8296806f80f4ef6b9b048b43da68af7', 'request_id': 'req-1ca1ec27-b382-4405-85e8-88244fa8f73a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.028610 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8296806f80f4ef6b9b048b43da68af7', 'request_id': 'req-1ca1ec27-b382-4405-85e8-88244fa8f73a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.034308 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8296806f80f4ef6b9b048b43da68af7', 'request_id': 'req-1ca1ec27-b382-4405-85e8-88244fa8f73a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.048575 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] The token cache is being invalidate because project a8296806f80f4ef6b9b048b43da68af7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:43.048869 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.049513 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.051019 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1425/2849] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:42 2022] DELETE /identity/v3/projects/a8296806f80f4ef6b9b048b43da68af7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:43.056655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d45cb-3805-4fe2-a7b2-c301a9660a2e service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:43.068437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d45cb-3805-4fe2-a7b2-c301a9660a2e service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:43.141547 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff6497-5361-40ef-a679-2647e764c893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:43.142660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff6497-5361-40ef-a679-2647e764c893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:43.142660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff6497-5361-40ef-a679-2647e764c893 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:43.142918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff6497-5361-40ef-a679-2647e764c893 admin admin] PATH_INFO: `/v3/users/588af476986a4ae2add9cb1408df6c0e` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:43.144259 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff6497-5361-40ef-a679-2647e764c893 admin admin] RBAC: Authorizing `identity:delete_user(user_id=588af476986a4ae2add9cb1408df6c0e)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:43.146269 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff6497-5361-40ef-a679-2647e764c893 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:43.197604 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-94ff6497-5361-40ef-a679-2647e764c893 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '588af476986a4ae2add9cb1408df6c0e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.213556 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-94ff6497-5361-40ef-a679-2647e764c893 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '588af476986a4ae2add9cb1408df6c0e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.214305 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-94ff6497-5361-40ef-a679-2647e764c893 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '588af476986a4ae2add9cb1408df6c0e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.214910 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-94ff6497-5361-40ef-a679-2647e764c893 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '588af476986a4ae2add9cb1408df6c0e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.222928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-94ff6497-5361-40ef-a679-2647e764c893 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '588af476986a4ae2add9cb1408df6c0e'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.237225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1425/2850] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:43 2022] DELETE /identity/v3/users/588af476986a4ae2add9cb1408df6c0e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:43.586937 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:43.588677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:43.624894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1ec27-b382-4405-85e8-88244fa8f73a admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:43.748189 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2565d44-4504-45bc-99a6-a78adae5ba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:43.749629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2565d44-4504-45bc-99a6-a78adae5ba3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:43.750072 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2565d44-4504-45bc-99a6-a78adae5ba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:43.750987 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2565d44-4504-45bc-99a6-a78adae5ba3c admin admin] PATH_INFO: `/v3/projects/c9fb5a0cd7b14b868184fc4f685dc270` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:43.752433 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2565d44-4504-45bc-99a6-a78adae5ba3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9fb5a0cd7b14b868184fc4f685dc270)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:43.755520 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2565d44-4504-45bc-99a6-a78adae5ba3c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:43.832028 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d2565d44-4504-45bc-99a6-a78adae5ba3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9fb5a0cd7b14b868184fc4f685dc270', 'request_id': 'req-d2565d44-4504-45bc-99a6-a78adae5ba3c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.839989 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d2565d44-4504-45bc-99a6-a78adae5ba3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9fb5a0cd7b14b868184fc4f685dc270', 'request_id': 'req-d2565d44-4504-45bc-99a6-a78adae5ba3c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.844233 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:43.848038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d2565d44-4504-45bc-99a6-a78adae5ba3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9fb5a0cd7b14b868184fc4f685dc270', 'request_id': 'req-d2565d44-4504-45bc-99a6-a78adae5ba3c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.873797 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d2565d44-4504-45bc-99a6-a78adae5ba3c admin admin] The token cache is being invalidate because project c9fb5a0cd7b14b868184fc4f685dc270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:43.874351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d2565d44-4504-45bc-99a6-a78adae5ba3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.875312 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d2565d44-4504-45bc-99a6-a78adae5ba3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:43.877865 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1426/2851] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:43 2022] DELETE /identity/v3/projects/c9fb5a0cd7b14b868184fc4f685dc270 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:43.883757 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978c5ea-8b20-4a2b-860f-733de6675162 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:43.884037 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978c5ea-8b20-4a2b-860f-733de6675162 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:43.884315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978c5ea-8b20-4a2b-860f-733de6675162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:43.910764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-c978c5ea-8b20-4a2b-860f-733de6675162 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:43.912382 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-c978c5ea-8b20-4a2b-860f-733de6675162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:43.983704 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1426/2852] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:43.991402 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419835d-6a84-4c55-b9b6-ab1fb88442eb None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:43.991685 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419835d-6a84-4c55-b9b6-ab1fb88442eb None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:43.991953 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419835d-6a84-4c55-b9b6-ab1fb88442eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.013313 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-9419835d-6a84-4c55-b9b6-ab1fb88442eb None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:44.014509 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9419835d-6a84-4c55-b9b6-ab1fb88442eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:44.022984 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1427/2853] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:44.027581 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978c5ea-8b20-4a2b-860f-733de6675162 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:44.032918 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:44.040189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978c5ea-8b20-4a2b-860f-733de6675162 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:44.046250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce585deb-10f0-4510-bc8d-fdaad3b25074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:44.046850 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce585deb-10f0-4510-bc8d-fdaad3b25074 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.047160 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce585deb-10f0-4510-bc8d-fdaad3b25074 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.047592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce585deb-10f0-4510-bc8d-fdaad3b25074 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.048263 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce585deb-10f0-4510-bc8d-fdaad3b25074 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:57:44.048563 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce585deb-10f0-4510-bc8d-fdaad3b25074 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:44.050136 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce585deb-10f0-4510-bc8d-fdaad3b25074 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:44.062437 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1427/2854] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:57:44.066813 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419835d-6a84-4c55-b9b6-ab1fb88442eb None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:44.077214 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419835d-6a84-4c55-b9b6-ab1fb88442eb None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:44.081928 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219fd5b-00ab-4b2e-95a1-076ddf10309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:44.082517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219fd5b-00ab-4b2e-95a1-076ddf10309b admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.082796 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219fd5b-00ab-4b2e-95a1-076ddf10309b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.083068 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219fd5b-00ab-4b2e-95a1-076ddf10309b admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.083779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219fd5b-00ab-4b2e-95a1-076ddf10309b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:44.086669 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219fd5b-00ab-4b2e-95a1-076ddf10309b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:44.117417 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1428/2855] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:44 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:44.121991 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce585deb-10f0-4510-bc8d-fdaad3b25074 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:44.132967 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce585deb-10f0-4510-bc8d-fdaad3b25074 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:44.190216 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ca59b-44bf-4afe-88a4-53103a74d37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:44.190815 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ca59b-44bf-4afe-88a4-53103a74d37b admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.191093 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ca59b-44bf-4afe-88a4-53103a74d37b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.191393 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ca59b-44bf-4afe-88a4-53103a74d37b admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.192017 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ca59b-44bf-4afe-88a4-53103a74d37b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:44.194438 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ca59b-44bf-4afe-88a4-53103a74d37b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:44.218086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1428/2856] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:44 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:44.222350 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219fd5b-00ab-4b2e-95a1-076ddf10309b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:44.233216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219fd5b-00ab-4b2e-95a1-076ddf10309b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:44.237820 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237a275-c437-4cdf-8e68-679e335b21d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:44.238408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237a275-c437-4cdf-8e68-679e335b21d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.238719 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237a275-c437-4cdf-8e68-679e335b21d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.238991 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237a275-c437-4cdf-8e68-679e335b21d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.239666 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237a275-c437-4cdf-8e68-679e335b21d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:44.242244 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237a275-c437-4cdf-8e68-679e335b21d2 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:44.252607 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1429/2857] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:44.256838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ca59b-44bf-4afe-88a4-53103a74d37b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:44.267677 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ca59b-44bf-4afe-88a4-53103a74d37b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:44.272660 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82a70a-d4f6-4a82-ade9-8b4b4b82efef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:44.273277 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82a70a-d4f6-4a82-ade9-8b4b4b82efef admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.273779 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82a70a-d4f6-4a82-ade9-8b4b4b82efef admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.274112 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82a70a-d4f6-4a82-ade9-8b4b4b82efef admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.274761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82a70a-d4f6-4a82-ade9-8b4b4b82efef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:44.276400 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82a70a-d4f6-4a82-ade9-8b4b4b82efef admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:44.288076 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1429/2858] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:44.292880 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237a275-c437-4cdf-8e68-679e335b21d2 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:44.302844 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237a275-c437-4cdf-8e68-679e335b21d2 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:44.308236 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542fde3-997e-49bb-a8bc-47601033d8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:44.308881 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542fde3-997e-49bb-a8bc-47601033d8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.309158 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542fde3-997e-49bb-a8bc-47601033d8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.309429 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542fde3-997e-49bb-a8bc-47601033d8f1 admin admin] PATH_INFO: `/v3/projects/5e5b75782e544cdea9d723946fe1f52a/users/3111f4fbb623407caf1130a36886c293/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.324963 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542fde3-997e-49bb-a8bc-47601033d8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e5b75782e544cdea9d723946fe1f52a, user_id=3111f4fbb623407caf1130a36886c293, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:44.327725 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542fde3-997e-49bb-a8bc-47601033d8f1 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:44.347823 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1430/2859] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:44 2022] PUT /identity/v3/projects/5e5b75782e544cdea9d723946fe1f52a/users/3111f4fbb623407caf1130a36886c293/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:44.359521 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9f116-aa10-419c-a38d-e9dfea7641e7 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.359813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9f116-aa10-419c-a38d-e9dfea7641e7 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.360126 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9f116-aa10-419c-a38d-e9dfea7641e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.400200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e8a9f116-aa10-419c-a38d-e9dfea7641e7 None None] MFA Rules not processed for user `3111f4fbb623407caf1130a36886c293`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:44.401456 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e8a9f116-aa10-419c-a38d-e9dfea7641e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:44.473378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1430/2860] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:44.478653 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542fde3-997e-49bb-a8bc-47601033d8f1 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:44.546706 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542fde3-997e-49bb-a8bc-47601033d8f1 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:44.551384 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ba132-1937-411b-b690-0a19bdbed7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:44.552210 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ba132-1937-411b-b690-0a19bdbed7f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.552366 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ba132-1937-411b-b690-0a19bdbed7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.552511 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ba132-1937-411b-b690-0a19bdbed7f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.553174 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ba132-1937-411b-b690-0a19bdbed7f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:44.555408 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ba132-1937-411b-b690-0a19bdbed7f6 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:44.583342 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1431/2861] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:44 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:44.587820 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9f116-aa10-419c-a38d-e9dfea7641e7 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:44.598629 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9f116-aa10-419c-a38d-e9dfea7641e7 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:44.651770 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:44.659546 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67363458-9e15-4b07-8647-ad02b65c8acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:44.660316 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67363458-9e15-4b07-8647-ad02b65c8acd admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.660594 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67363458-9e15-4b07-8647-ad02b65c8acd admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.660863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67363458-9e15-4b07-8647-ad02b65c8acd admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.661541 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67363458-9e15-4b07-8647-ad02b65c8acd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:44.664820 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67363458-9e15-4b07-8647-ad02b65c8acd admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:44.686435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1431/2862] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:44 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:44.690456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ba132-1937-411b-b690-0a19bdbed7f6 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:44.701046 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ba132-1937-411b-b690-0a19bdbed7f6 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:44.705592 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17171e8-daad-4ef0-9000-c98bd9c14b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:44.706179 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17171e8-daad-4ef0-9000-c98bd9c14b39 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.706454 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17171e8-daad-4ef0-9000-c98bd9c14b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.706721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17171e8-daad-4ef0-9000-c98bd9c14b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.707383 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17171e8-daad-4ef0-9000-c98bd9c14b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:44.709853 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17171e8-daad-4ef0-9000-c98bd9c14b39 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:44.720684 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1432/2863] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:44.725786 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67363458-9e15-4b07-8647-ad02b65c8acd admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:44.736095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67363458-9e15-4b07-8647-ad02b65c8acd admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:44.742664 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f41d8b-612f-491f-9408-0033945c1204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:44.743607 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f41d8b-612f-491f-9408-0033945c1204 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.743882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f41d8b-612f-491f-9408-0033945c1204 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.744148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f41d8b-612f-491f-9408-0033945c1204 admin admin] PATH_INFO: `/v3/projects/0753b0ca7df6442dae4b2890b6294149/users/99a22158e1a04a01a27b0fe2d39bbe92/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.759896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f41d8b-612f-491f-9408-0033945c1204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0753b0ca7df6442dae4b2890b6294149, user_id=99a22158e1a04a01a27b0fe2d39bbe92, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:44.762513 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f41d8b-612f-491f-9408-0033945c1204 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:44.776611 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1432/2864] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:44 2022] PUT /identity/v3/projects/0753b0ca7df6442dae4b2890b6294149/users/99a22158e1a04a01a27b0fe2d39bbe92/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:44.780812 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17171e8-daad-4ef0-9000-c98bd9c14b39 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:44.791757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17171e8-daad-4ef0-9000-c98bd9c14b39 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:44.852419 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:44.860099 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182852df-06c5-4c5c-a0f4-e2cd35e04001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:44.861071 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182852df-06c5-4c5c-a0f4-e2cd35e04001 admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.861548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182852df-06c5-4c5c-a0f4-e2cd35e04001 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.861838 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182852df-06c5-4c5c-a0f4-e2cd35e04001 admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.862500 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182852df-06c5-4c5c-a0f4-e2cd35e04001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:44.864591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182852df-06c5-4c5c-a0f4-e2cd35e04001 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:44.876431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1433/2865] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:44.881135 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f41d8b-612f-491f-9408-0033945c1204 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:44.891631 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f41d8b-612f-491f-9408-0033945c1204 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:44.896178 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54be6c6-f8a4-4450-9d83-e102e744a9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:44.896852 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54be6c6-f8a4-4450-9d83-e102e744a9c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.897326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54be6c6-f8a4-4450-9d83-e102e744a9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.897838 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54be6c6-f8a4-4450-9d83-e102e744a9c3 admin admin] PATH_INFO: `/v3/projects/0753b0ca7df6442dae4b2890b6294149/users/99a22158e1a04a01a27b0fe2d39bbe92/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.899679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54be6c6-f8a4-4450-9d83-e102e744a9c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0753b0ca7df6442dae4b2890b6294149, user_id=99a22158e1a04a01a27b0fe2d39bbe92, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:44.902403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54be6c6-f8a4-4450-9d83-e102e744a9c3 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:44.913844 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1433/2866] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:44 2022] PUT /identity/v3/projects/0753b0ca7df6442dae4b2890b6294149/users/99a22158e1a04a01a27b0fe2d39bbe92/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:44.925305 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451d25-76b3-4dcd-bd1f-e11584781120 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.925591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451d25-76b3-4dcd-bd1f-e11584781120 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.925866 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451d25-76b3-4dcd-bd1f-e11584781120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.959216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b4451d25-76b3-4dcd-bd1f-e11584781120 None None] MFA Rules not processed for user `99a22158e1a04a01a27b0fe2d39bbe92`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:44.961935 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b4451d25-76b3-4dcd-bd1f-e11584781120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:45.038612 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1434/2867] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:45.049330 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54be6c6-f8a4-4450-9d83-e102e744a9c3 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:45.051054 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d54be6c6-f8a4-4450-9d83-e102e744a9c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:45.118478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54be6c6-f8a4-4450-9d83-e102e744a9c3 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:45.122318 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb378313-1055-444d-ba2e-81272f050668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d8ade66b7e4a1880c961c5e13adf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:45.122935 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb378313-1055-444d-ba2e-81272f050668 service cinder] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:45.123243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb378313-1055-444d-ba2e-81272f050668 service cinder] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:45.123519 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb378313-1055-444d-ba2e-81272f050668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:45.133723 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb378313-1055-444d-ba2e-81272f050668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:45.135189 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb378313-1055-444d-ba2e-81272f050668 service cinder] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:45.144570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1434/2868] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:45.545267 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451d25-76b3-4dcd-bd1f-e11584781120 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:45.618550 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451d25-76b3-4dcd-bd1f-e11584781120 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:45.627025 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:45.627764 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:45.628183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:45.628572 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] PATH_INFO: `/v3/users/3111f4fbb623407caf1130a36886c293` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:45.629971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3111f4fbb623407caf1130a36886c293)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:45.632423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:45.692960 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3111f4fbb623407caf1130a36886c293'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:45.702711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3111f4fbb623407caf1130a36886c293'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:45.703567 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3111f4fbb623407caf1130a36886c293'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:45.718779 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3111f4fbb623407caf1130a36886c293'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:45.719536 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3111f4fbb623407caf1130a36886c293'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:45.728729 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:45.735678 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1435/2869] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:45 2022] DELETE /identity/v3/users/3111f4fbb623407caf1130a36886c293 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:45.742148 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64d359-54bc-4828-bf29-2f48616365e4 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:45.743355 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64d359-54bc-4828-bf29-2f48616365e4 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:45.743524 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64d359-54bc-4828-bf29-2f48616365e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:45.765228 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-4c64d359-54bc-4828-bf29-2f48616365e4 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:45.766579 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4c64d359-54bc-4828-bf29-2f48616365e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:45.844843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1435/2870] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:45.857679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:45.859303 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:45.931921 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcb3e2-c6b8-4bb8-bf6b-5b8b64795339 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:45.935589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fddf4a-7f34-41df-89ee-e0e870e22768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:45.936194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fddf4a-7f34-41df-89ee-e0e870e22768 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:45.936474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fddf4a-7f34-41df-89ee-e0e870e22768 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:45.936742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fddf4a-7f34-41df-89ee-e0e870e22768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:45.946400 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fddf4a-7f34-41df-89ee-e0e870e22768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:45.947816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fddf4a-7f34-41df-89ee-e0e870e22768 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:45.958300 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1436/2871] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:46.253327 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64d359-54bc-4828-bf29-2f48616365e4 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:46.254890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4c64d359-54bc-4828-bf29-2f48616365e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:46.268471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64d359-54bc-4828-bf29-2f48616365e4 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:46.273992 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:46.274657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:46.274942 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:46.275321 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] PATH_INFO: `/v3/projects/5e5b75782e544cdea9d723946fe1f52a` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:46.276676 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e5b75782e544cdea9d723946fe1f52a)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:46.278509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:46.333060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:46.335334 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e5b75782e544cdea9d723946fe1f52a', 'request_id': 'req-69c80524-4e33-4884-8a77-9f3d6a32070d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:46.346185 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e5b75782e544cdea9d723946fe1f52a', 'request_id': 'req-69c80524-4e33-4884-8a77-9f3d6a32070d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:46.354695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e5b75782e544cdea9d723946fe1f52a', 'request_id': 'req-69c80524-4e33-4884-8a77-9f3d6a32070d'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:46.366349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] The token cache is being invalidate because project 5e5b75782e544cdea9d723946fe1f52a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:46.366653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:46.367348 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:46.369183 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1436/2872] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:46 2022] DELETE /identity/v3/projects/5e5b75782e544cdea9d723946fe1f52a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:46.373446 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fddf4a-7f34-41df-89ee-e0e870e22768 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:46.385548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fddf4a-7f34-41df-89ee-e0e870e22768 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:46.451626 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:46.452272 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:46.452552 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:46.452832 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] PATH_INFO: `/v3/users/99a22158e1a04a01a27b0fe2d39bbe92` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:46.454003 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99a22158e1a04a01a27b0fe2d39bbe92)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:46.456061 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:46.517091 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99a22158e1a04a01a27b0fe2d39bbe92'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:46.524952 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99a22158e1a04a01a27b0fe2d39bbe92'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:46.525625 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99a22158e1a04a01a27b0fe2d39bbe92'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:46.540742 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99a22158e1a04a01a27b0fe2d39bbe92'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:46.541522 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99a22158e1a04a01a27b0fe2d39bbe92'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:46.555420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1437/2873] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:46 2022] DELETE /identity/v3/users/99a22158e1a04a01a27b0fe2d39bbe92 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:46.848377 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:46.853087 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:46.874045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c80524-4e33-4884-8a77-9f3d6a32070d admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:46.957471 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:46.958069 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:46.958303 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:46.958693 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] PATH_INFO: `/v3/projects/0753b0ca7df6442dae4b2890b6294149` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:46.960538 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0753b0ca7df6442dae4b2890b6294149)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:46.962679 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:47.022802 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0753b0ca7df6442dae4b2890b6294149', 'request_id': 'req-622fcd83-3e7c-44cc-8a5d-2312d83b973a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:47.030539 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0753b0ca7df6442dae4b2890b6294149', 'request_id': 'req-622fcd83-3e7c-44cc-8a5d-2312d83b973a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:47.037158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0753b0ca7df6442dae4b2890b6294149', 'request_id': 'req-622fcd83-3e7c-44cc-8a5d-2312d83b973a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:47.049516 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] The token cache is being invalidate because project 0753b0ca7df6442dae4b2890b6294149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:47.049813 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:47.050487 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:47.052415 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1437/2874] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:46 2022] DELETE /identity/v3/projects/0753b0ca7df6442dae4b2890b6294149 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:56.551009 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=80,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:57.046084 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=58,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:21.881645 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:21.883140 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:21.944216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfae1d7-4d5f-421c-b769-b5f2941841e1 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:21.997388 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bb206-61aa-43ef-af7f-b23c5be3668c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:21.997986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bb206-61aa-43ef-af7f-b23c5be3668c service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:21.998264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bb206-61aa-43ef-af7f-b23c5be3668c service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:21.998534 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bb206-61aa-43ef-af7f-b23c5be3668c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:22.000298 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-de7bb206-61aa-43ef-af7f-b23c5be3668c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:22.067100 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bb206-61aa-43ef-af7f-b23c5be3668c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:22.068560 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bb206-61aa-43ef-af7f-b23c5be3668c service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:22.078748 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1438/2875] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:30.286282 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:30.289077 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:30.381020 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fcd83-3e7c-44cc-8a5d-2312d83b973a admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:30.393657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716dd2c-78a0-423a-8fa8-cc4d0887952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:30.394578 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716dd2c-78a0-423a-8fa8-cc4d0887952d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:30.394971 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716dd2c-78a0-423a-8fa8-cc4d0887952d admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:30.395419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716dd2c-78a0-423a-8fa8-cc4d0887952d admin admin] PATH_INFO: `/v3/users/488e41d766134f3aa7c956816d01df06` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:30.397914 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716dd2c-78a0-423a-8fa8-cc4d0887952d admin admin] RBAC: Authorizing `identity:delete_user(user_id=488e41d766134f3aa7c956816d01df06)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:30.400434 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716dd2c-78a0-423a-8fa8-cc4d0887952d admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:30.469141 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1716dd2c-78a0-423a-8fa8-cc4d0887952d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '488e41d766134f3aa7c956816d01df06'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:30.488725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1716dd2c-78a0-423a-8fa8-cc4d0887952d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '488e41d766134f3aa7c956816d01df06'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:30.489726 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1716dd2c-78a0-423a-8fa8-cc4d0887952d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '488e41d766134f3aa7c956816d01df06'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:30.490569 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1716dd2c-78a0-423a-8fa8-cc4d0887952d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '488e41d766134f3aa7c956816d01df06'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:30.517761 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-1716dd2c-78a0-423a-8fa8-cc4d0887952d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '488e41d766134f3aa7c956816d01df06'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:30.532794 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1438/2876] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:58:30 2022] DELETE /identity/v3/users/488e41d766134f3aa7c956816d01df06 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:30.795906 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bb206-61aa-43ef-af7f-b23c5be3668c service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:30.813855 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bb206-61aa-43ef-af7f-b23c5be3668c service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:30.919727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef9d3b-9030-41dd-9792-b7136a48980b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:30.920365 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef9d3b-9030-41dd-9792-b7136a48980b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:30.920640 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef9d3b-9030-41dd-9792-b7136a48980b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:30.920911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef9d3b-9030-41dd-9792-b7136a48980b admin admin] PATH_INFO: `/v3/projects/fbdd47c941404548bf5371c35e518346` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:30.922114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef9d3b-9030-41dd-9792-b7136a48980b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbdd47c941404548bf5371c35e518346)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:30.924131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef9d3b-9030-41dd-9792-b7136a48980b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:30.934315 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:31.008544 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-deef9d3b-9030-41dd-9792-b7136a48980b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbdd47c941404548bf5371c35e518346', 'request_id': 'req-deef9d3b-9030-41dd-9792-b7136a48980b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:31.016548 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-deef9d3b-9030-41dd-9792-b7136a48980b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbdd47c941404548bf5371c35e518346', 'request_id': 'req-deef9d3b-9030-41dd-9792-b7136a48980b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:31.024539 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-deef9d3b-9030-41dd-9792-b7136a48980b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbdd47c941404548bf5371c35e518346', 'request_id': 'req-deef9d3b-9030-41dd-9792-b7136a48980b'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:31.040129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-deef9d3b-9030-41dd-9792-b7136a48980b admin admin] The token cache is being invalidate because project fbdd47c941404548bf5371c35e518346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:58:31.040517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-deef9d3b-9030-41dd-9792-b7136a48980b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:31.041333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-deef9d3b-9030-41dd-9792-b7136a48980b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:31.042988 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1439/2877] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:58:30 2022] DELETE /identity/v3/projects/fbdd47c941404548bf5371c35e518346 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:31.049304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48d1ac-79f7-438b-8228-f6dbf914023a None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:31.049823 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48d1ac-79f7-438b-8228-f6dbf914023a None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:31.050233 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48d1ac-79f7-438b-8228-f6dbf914023a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:31.076626 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-0c48d1ac-79f7-438b-8228-f6dbf914023a None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:31.077900 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-0c48d1ac-79f7-438b-8228-f6dbf914023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:31.150113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1439/2878] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:31.159711 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc156fb4-e803-4131-9e7c-42a2e02ddaa3 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:31.160181 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc156fb4-e803-4131-9e7c-42a2e02ddaa3 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:31.160650 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc156fb4-e803-4131-9e7c-42a2e02ddaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:31.191547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-dc156fb4-e803-4131-9e7c-42a2e02ddaa3 None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:31.192677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-dc156fb4-e803-4131-9e7c-42a2e02ddaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:31.201346 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1440/2879] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:31.206039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48d1ac-79f7-438b-8228-f6dbf914023a None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:31.218843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48d1ac-79f7-438b-8228-f6dbf914023a None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:31.224476 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee4ba7-9b3c-43d9-80cc-2251c0baff80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:31.225096 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee4ba7-9b3c-43d9-80cc-2251c0baff80 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:31.225967 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee4ba7-9b3c-43d9-80cc-2251c0baff80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:31.226695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee4ba7-9b3c-43d9-80cc-2251c0baff80 admin admin] PATH_INFO: `/v3/domains` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:31.226924 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee4ba7-9b3c-43d9-80cc-2251c0baff80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:58:31.227272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee4ba7-9b3c-43d9-80cc-2251c0baff80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:31.228916 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee4ba7-9b3c-43d9-80cc-2251c0baff80 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:31.243997 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1440/2880] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 09:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:58:31.250646 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc156fb4-e803-4131-9e7c-42a2e02ddaa3 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:31.264160 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc156fb4-e803-4131-9e7c-42a2e02ddaa3 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:31.271341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58448247-1457-479f-8878-bf663349e7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:31.271962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58448247-1457-479f-8878-bf663349e7ec admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:31.272858 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58448247-1457-479f-8878-bf663349e7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:31.273333 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58448247-1457-479f-8878-bf663349e7ec admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:31.274629 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58448247-1457-479f-8878-bf663349e7ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:31.278114 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58448247-1457-479f-8878-bf663349e7ec admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:31.338129 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1441/2881] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:31 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:31.346570 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee4ba7-9b3c-43d9-80cc-2251c0baff80 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:31.358168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee4ba7-9b3c-43d9-80cc-2251c0baff80 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:31.363402 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:31.440689 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a1d84-17b1-405b-9aa6-4fd90e857f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:31.441315 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a1d84-17b1-405b-9aa6-4fd90e857f2b admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:31.441642 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a1d84-17b1-405b-9aa6-4fd90e857f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:31.441862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a1d84-17b1-405b-9aa6-4fd90e857f2b admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:31.442510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a1d84-17b1-405b-9aa6-4fd90e857f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:31.445058 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a1d84-17b1-405b-9aa6-4fd90e857f2b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:31.480344 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1441/2882] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:31 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:31.485183 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58448247-1457-479f-8878-bf663349e7ec admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:31.501393 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58448247-1457-479f-8878-bf663349e7ec admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:31.506404 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92af4b-2e3a-4a58-b4d1-b1d170ad3d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:31.507573 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92af4b-2e3a-4a58-b4d1-b1d170ad3d3c admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:31.507857 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92af4b-2e3a-4a58-b4d1-b1d170ad3d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:31.508122 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92af4b-2e3a-4a58-b4d1-b1d170ad3d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:31.508778 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92af4b-2e3a-4a58-b4d1-b1d170ad3d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:31.510406 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92af4b-2e3a-4a58-b4d1-b1d170ad3d3c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:31.525917 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1442/2883] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:58:31.531168 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a1d84-17b1-405b-9aa6-4fd90e857f2b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:31.545333 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a1d84-17b1-405b-9aa6-4fd90e857f2b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:31.551553 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7357c7-4042-4421-ab76-5b38726d2f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:31.552545 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7357c7-4042-4421-ab76-5b38726d2f84 admin admin] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:31.552731 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7357c7-4042-4421-ab76-5b38726d2f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:31.552896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7357c7-4042-4421-ab76-5b38726d2f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:31.554262 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7357c7-4042-4421-ab76-5b38726d2f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:31.557688 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7357c7-4042-4421-ab76-5b38726d2f84 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:31.577245 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1442/2884] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:31.581800 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92af4b-2e3a-4a58-b4d1-b1d170ad3d3c admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:31.595916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92af4b-2e3a-4a58-b4d1-b1d170ad3d3c admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:31.602621 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae75da-bac5-49d6-8839-132b46ffc749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:31.603486 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae75da-bac5-49d6-8839-132b46ffc749 admin admin] REQUEST_METHOD: `PUT` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:31.603806 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae75da-bac5-49d6-8839-132b46ffc749 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:31.604117 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae75da-bac5-49d6-8839-132b46ffc749 admin admin] PATH_INFO: `/v3/projects/3ad415e35f8746ff86bd0d4f96c3ca9e/users/f6393be1efdd4c12bcf3adbb230c4943/roles/7ccf7639a4864595b7bc589a9dc44df7` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:31.623040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae75da-bac5-49d6-8839-132b46ffc749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ad415e35f8746ff86bd0d4f96c3ca9e, user_id=f6393be1efdd4c12bcf3adbb230c4943, role_id=7ccf7639a4864595b7bc589a9dc44df7)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:31.626042 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae75da-bac5-49d6-8839-132b46ffc749 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:31.643733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1443/2885] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:31 2022] PUT /identity/v3/projects/3ad415e35f8746ff86bd0d4f96c3ca9e/users/f6393be1efdd4c12bcf3adbb230c4943/roles/7ccf7639a4864595b7bc589a9dc44df7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:58:31.653239 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510aec2-09dc-44fe-8f0f-d4242866a4ec None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:31.653717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510aec2-09dc-44fe-8f0f-d4242866a4ec None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:31.654134 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510aec2-09dc-44fe-8f0f-d4242866a4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:31.676417 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-e510aec2-09dc-44fe-8f0f-d4242866a4ec None None] MFA Rules not processed for user `d7063e9478d747e983e7ced895545d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:31.677665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e510aec2-09dc-44fe-8f0f-d4242866a4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:31.750740 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1443/2886] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:31.761411 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae75da-bac5-49d6-8839-132b46ffc749 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:31.763339 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-02ae75da-bac5-49d6-8839-132b46ffc749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:31.844937 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae75da-bac5-49d6-8839-132b46ffc749 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:31.848916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d04e8a-5e49-4f44-8131-96088033c508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:31.849538 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d04e8a-5e49-4f44-8131-96088033c508 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:31.849814 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d04e8a-5e49-4f44-8131-96088033c508 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:31.850082 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d04e8a-5e49-4f44-8131-96088033c508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:31.863036 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d04e8a-5e49-4f44-8131-96088033c508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:31.864527 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d04e8a-5e49-4f44-8131-96088033c508 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:31.879038 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1444/2887] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:37.478059 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c10822-c9e5-41bd-aba0-83dcdd7db675 None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:37.478571 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c10822-c9e5-41bd-aba0-83dcdd7db675 None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:37.478950 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c10822-c9e5-41bd-aba0-83dcdd7db675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:37.524497 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-81c10822-c9e5-41bd-aba0-83dcdd7db675 None None] MFA Rules not processed for user `f6393be1efdd4c12bcf3adbb230c4943`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:37.525862 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-81c10822-c9e5-41bd-aba0-83dcdd7db675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:37.611149 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1444/2888] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:58:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:37.617497 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d04e8a-5e49-4f44-8131-96088033c508 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:37.629732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d04e8a-5e49-4f44-8131-96088033c508 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:37.636146 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8f706-d2b3-4d22-a21a-4b4fe7bf67f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:37.636786 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8f706-d2b3-4d22-a21a-4b4fe7bf67f7 admin admin] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:37.637271 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8f706-d2b3-4d22-a21a-4b4fe7bf67f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:37.637545 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8f706-d2b3-4d22-a21a-4b4fe7bf67f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:37.638475 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8f706-d2b3-4d22-a21a-4b4fe7bf67f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:37.640958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8f706-d2b3-4d22-a21a-4b4fe7bf67f7 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:37.679197 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1445/2889] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:37 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:37.681278 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c10822-c9e5-41bd-aba0-83dcdd7db675 None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:37.693021 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c10822-c9e5-41bd-aba0-83dcdd7db675 None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:37.709285 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:37.785686 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08867df2-1793-49fd-a29b-2027058ae486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:37.786326 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867df2-1793-49fd-a29b-2027058ae486 admin admin] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:37.786600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867df2-1793-49fd-a29b-2027058ae486 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:37.786874 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867df2-1793-49fd-a29b-2027058ae486 admin admin] PATH_INFO: `/v3/users` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:37.787709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08867df2-1793-49fd-a29b-2027058ae486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:37.790293 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08867df2-1793-49fd-a29b-2027058ae486 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:37.820359 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1445/2890] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:37 2022] POST /identity/v3/users => generated 297 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:37.824596 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8f706-d2b3-4d22-a21a-4b4fe7bf67f7 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:37.833468 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:37.838268 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8f706-d2b3-4d22-a21a-4b4fe7bf67f7 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:37.845260 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813093a-1918-4063-a692-b4342c3ca33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:37.846622 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813093a-1918-4063-a692-b4342c3ca33e admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:37.847811 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813093a-1918-4063-a692-b4342c3ca33e admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:37.849746 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813093a-1918-4063-a692-b4342c3ca33e admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:37.850837 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813093a-1918-4063-a692-b4342c3ca33e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:37.852740 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813093a-1918-4063-a692-b4342c3ca33e admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:37.867292 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1446/2891] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:37.871863 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08867df2-1793-49fd-a29b-2027058ae486 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:37.888702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08867df2-1793-49fd-a29b-2027058ae486 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:37.897665 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da51a2-a9fd-478f-900d-b0a189333e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:37.898418 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da51a2-a9fd-478f-900d-b0a189333e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:37.898879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da51a2-a9fd-478f-900d-b0a189333e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:37.899201 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da51a2-a9fd-478f-900d-b0a189333e09 admin admin] PATH_INFO: `/v3/projects/9101f0890a934a4baee830f365d9eeaf/users/d00fe4ef34724b2bbe1870c82f4ffb3f/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:37.917956 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da51a2-a9fd-478f-900d-b0a189333e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9101f0890a934a4baee830f365d9eeaf, user_id=d00fe4ef34724b2bbe1870c82f4ffb3f, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:37.920977 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da51a2-a9fd-478f-900d-b0a189333e09 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:37.939627 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1446/2892] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:37 2022] PUT /identity/v3/projects/9101f0890a934a4baee830f365d9eeaf/users/d00fe4ef34724b2bbe1870c82f4ffb3f/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:37.944528 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813093a-1918-4063-a692-b4342c3ca33e admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:37.956010 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813093a-1918-4063-a692-b4342c3ca33e admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:38.026971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90eff9d-68f2-40b0-b564-7441440fc88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:38.027624 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90eff9d-68f2-40b0-b564-7441440fc88c admin admin] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:38.027904 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90eff9d-68f2-40b0-b564-7441440fc88c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:38.028177 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90eff9d-68f2-40b0-b564-7441440fc88c admin admin] PATH_INFO: `/v3/roles` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:38.028857 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90eff9d-68f2-40b0-b564-7441440fc88c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:38.030471 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90eff9d-68f2-40b0-b564-7441440fc88c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:38.041801 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1447/2893] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:38.046837 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da51a2-a9fd-478f-900d-b0a189333e09 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:38.063403 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da51a2-a9fd-478f-900d-b0a189333e09 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:38.068270 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84de69-86ac-402a-8034-9ed468d0537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:38.069506 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84de69-86ac-402a-8034-9ed468d0537a admin admin] REQUEST_METHOD: `PUT` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:38.069789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84de69-86ac-402a-8034-9ed468d0537a admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:38.070060 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84de69-86ac-402a-8034-9ed468d0537a admin admin] PATH_INFO: `/v3/projects/9101f0890a934a4baee830f365d9eeaf/users/d00fe4ef34724b2bbe1870c82f4ffb3f/roles/e7c31e56309e4f3880ee08fc63752e5d` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:38.072589 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84de69-86ac-402a-8034-9ed468d0537a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9101f0890a934a4baee830f365d9eeaf, user_id=d00fe4ef34724b2bbe1870c82f4ffb3f, role_id=e7c31e56309e4f3880ee08fc63752e5d)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:38.077066 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84de69-86ac-402a-8034-9ed468d0537a admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:38.090452 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1447/2894] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:38 2022] PUT /identity/v3/projects/9101f0890a934a4baee830f365d9eeaf/users/d00fe4ef34724b2bbe1870c82f4ffb3f/roles/e7c31e56309e4f3880ee08fc63752e5d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:44.023679 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02065af-eec0-4754-baf7-452cf0ff4759 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:44.024757 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02065af-eec0-4754-baf7-452cf0ff4759 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:44.025040 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02065af-eec0-4754-baf7-452cf0ff4759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:44.074735 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-c02065af-eec0-4754-baf7-452cf0ff4759 None None] MFA Rules not processed for user `d00fe4ef34724b2bbe1870c82f4ffb3f`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:44.076546 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c02065af-eec0-4754-baf7-452cf0ff4759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:44.181416 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1448/2895] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:58:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:44.200923 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84de69-86ac-402a-8034-9ed468d0537a admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:44.205103 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-2f84de69-86ac-402a-8034-9ed468d0537a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:44.276414 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84de69-86ac-402a-8034-9ed468d0537a admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:44.280623 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5ce01-29f6-49c0-b5fd-6d434049ed54 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06fad9466df14827983dac2b4b844f3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:44.281243 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5ce01-29f6-49c0-b5fd-6d434049ed54 service glance] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:44.281522 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5ce01-29f6-49c0-b5fd-6d434049ed54 service glance] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:44.281908 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5ce01-29f6-49c0-b5fd-6d434049ed54 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:44.350559 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5ce01-29f6-49c0-b5fd-6d434049ed54 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:44.352781 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5ce01-29f6-49c0-b5fd-6d434049ed54 service glance] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:44.363754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1448/2896] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:44.521846 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77d557-e3a1-4609-ba56-7e942c289f9a None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:44.522448 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77d557-e3a1-4609-ba56-7e942c289f9a None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:44.522967 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77d557-e3a1-4609-ba56-7e942c289f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:44.708770 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-1b77d557-e3a1-4609-ba56-7e942c289f9a None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:44.710708 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1b77d557-e3a1-4609-ba56-7e942c289f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:44.784971 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1449/2897] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:58:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:44.850479 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5ce01-29f6-49c0-b5fd-6d434049ed54 service glance] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:44.852717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-65e5ce01-29f6-49c0-b5fd-6d434049ed54 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:44.885504 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:44.948808 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5ce01-29f6-49c0-b5fd-6d434049ed54 service glance] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:44.954571 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15726b8c-4874-47f5-b65a-29bd9143fa2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:44.955225 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15726b8c-4874-47f5-b65a-29bd9143fa2f service swift] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:44.955511 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15726b8c-4874-47f5-b65a-29bd9143fa2f service swift] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:44.955798 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15726b8c-4874-47f5-b65a-29bd9143fa2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:44.967098 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15726b8c-4874-47f5-b65a-29bd9143fa2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:44.968831 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15726b8c-4874-47f5-b65a-29bd9143fa2f service swift] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:44.979073 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1449/2898] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:58:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:58:54.414893 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba24e82-620f-4678-8f96-b84671bb881d None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:54.415586 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba24e82-620f-4678-8f96-b84671bb881d None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:54.415586 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba24e82-620f-4678-8f96-b84671bb881d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:54.436929 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-3ba24e82-620f-4678-8f96-b84671bb881d None None] MFA Rules not processed for user `f6393be1efdd4c12bcf3adbb230c4943`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:54.438150 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3ba24e82-620f-4678-8f96-b84671bb881d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:54.446579 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1450/2899] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 09:58:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:54.493102 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15726b8c-4874-47f5-b65a-29bd9143fa2f service swift] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:54.510426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15726b8c-4874-47f5-b65a-29bd9143fa2f service swift] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:54.516712 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d8be5-8d5b-4b69-8dde-e1ba25d4849f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06fad9466df14827983dac2b4b844f3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:54.517751 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d8be5-8d5b-4b69-8dde-e1ba25d4849f service glance] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:54.518250 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d8be5-8d5b-4b69-8dde-e1ba25d4849f service glance] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:54.518742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d8be5-8d5b-4b69-8dde-e1ba25d4849f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:54.522464 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-963d8be5-8d5b-4b69-8dde-e1ba25d4849f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:54.652466 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d8be5-8d5b-4b69-8dde-e1ba25d4849f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:54.655294 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d8be5-8d5b-4b69-8dde-e1ba25d4849f service glance] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:54.676987 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1450/2900] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:55.326847 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba24e82-620f-4678-8f96-b84671bb881d None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:55.329431 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3ba24e82-620f-4678-8f96-b84671bb881d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:55.334220 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d8be5-8d5b-4b69-8dde-e1ba25d4849f service glance] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:55.343900 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=100 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:55.413552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d8be5-8d5b-4b69-8dde-e1ba25d4849f service glance] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:55.419966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439ac9f-1bb8-43e4-97f0-4b9f6a4d5eb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:55.421230 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439ac9f-1bb8-43e4-97f0-4b9f6a4d5eb6 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:55.421643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439ac9f-1bb8-43e4-97f0-4b9f6a4d5eb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:55.422137 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439ac9f-1bb8-43e4-97f0-4b9f6a4d5eb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:55.423450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba24e82-620f-4678-8f96-b84671bb881d None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:55.426525 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-a439ac9f-1bb8-43e4-97f0-4b9f6a4d5eb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:55.430449 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985412c-f369-4e18-9324-5bf5099a09cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:55.431252 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985412c-f369-4e18-9324-5bf5099a09cd service placement] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:55.431591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985412c-f369-4e18-9324-5bf5099a09cd service placement] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:55.431932 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985412c-f369-4e18-9324-5bf5099a09cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:55.433923 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-1985412c-f369-4e18-9324-5bf5099a09cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:55.447330 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985412c-f369-4e18-9324-5bf5099a09cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:55.449031 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985412c-f369-4e18-9324-5bf5099a09cd service placement] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:55.455742 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439ac9f-1bb8-43e4-97f0-4b9f6a4d5eb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:55.457717 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439ac9f-1bb8-43e4-97f0-4b9f6a4d5eb6 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:55.462467 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1451/2901] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:55.488478 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1451/2902] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:58:57.150908 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4da1a-67d7-4983-bc27-e82babaf26e8 None None] REQUEST_METHOD: `POST` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:57.151571 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4da1a-67d7-4983-bc27-e82babaf26e8 None None] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:57.151571 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4da1a-67d7-4983-bc27-e82babaf26e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:57.218677 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.auth.core [None req-b6f4da1a-67d7-4983-bc27-e82babaf26e8 None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:57.220023 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-b6f4da1a-67d7-4983-bc27-e82babaf26e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:57.246345 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1452/2903] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 09:58:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:57.261743 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439ac9f-1bb8-43e4-97f0-4b9f6a4d5eb6 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:57.284378 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439ac9f-1bb8-43e4-97f0-4b9f6a4d5eb6 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:57.290439 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8740d2-8cad-415f-acb8-55ccc674d8e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:57.291063 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8740d2-8cad-415f-acb8-55ccc674d8e4 service swift] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:57.291368 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8740d2-8cad-415f-acb8-55ccc674d8e4 service swift] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:57.291634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8740d2-8cad-415f-acb8-55ccc674d8e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:57.300507 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8740d2-8cad-415f-acb8-55ccc674d8e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:57.302145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8740d2-8cad-415f-acb8-55ccc674d8e4 service swift] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:57.311766 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1452/2904] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 09:58:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:58:58.083245 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4da1a-67d7-4983-bc27-e82babaf26e8 None None] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:58.094101 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4da1a-67d7-4983-bc27-e82babaf26e8 None None] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:58.097943 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7897aba-dba2-424f-8691-162f7909c1bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:58.098614 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7897aba-dba2-424f-8691-162f7909c1bb service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:58.098901 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7897aba-dba2-424f-8691-162f7909c1bb service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:58.099221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7897aba-dba2-424f-8691-162f7909c1bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:58.102087 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f7897aba-dba2-424f-8691-162f7909c1bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:58.113751 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7897aba-dba2-424f-8691-162f7909c1bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:58.115332 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7897aba-dba2-424f-8691-162f7909c1bb service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:58.124894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1453/2905] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:07.308105 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=92 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:08.120562 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=54 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:09.775430 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8740d2-8cad-415f-acb8-55ccc674d8e4 service swift] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.784818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-1e8740d2-8cad-415f-acb8-55ccc674d8e4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:09.807896 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8740d2-8cad-415f-acb8-55ccc674d8e4 service swift] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.826843 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fab96c-b9af-4511-a6e9-bace9c315e3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.830253 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fab96c-b9af-4511-a6e9-bace9c315e3b service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.831349 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fab96c-b9af-4511-a6e9-bace9c315e3b service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.832032 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fab96c-b9af-4511-a6e9-bace9c315e3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.835787 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-86fab96c-b9af-4511-a6e9-bace9c315e3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:09.851789 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fab96c-b9af-4511-a6e9-bace9c315e3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.853723 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fab96c-b9af-4511-a6e9-bace9c315e3b service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.870453 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1453/2906] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:19.862045 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=6 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:23.070727 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7897aba-dba2-424f-8691-162f7909c1bb service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:23.079648 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7897aba-dba2-424f-8691-162f7909c1bb service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:23.083460 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd92cb-101c-4f59-b20a-a9deae48b4b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:23.084056 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd92cb-101c-4f59-b20a-a9deae48b4b0 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:23.085064 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd92cb-101c-4f59-b20a-a9deae48b4b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:23.085338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd92cb-101c-4f59-b20a-a9deae48b4b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:23.087047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3bdd92cb-101c-4f59-b20a-a9deae48b4b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:23.153638 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd92cb-101c-4f59-b20a-a9deae48b4b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:23.155264 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd92cb-101c-4f59-b20a-a9deae48b4b0 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:23.165894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1454/2907] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:26.410016 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fab96c-b9af-4511-a6e9-bace9c315e3b service nova] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:26.443004 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fab96c-b9af-4511-a6e9-bace9c315e3b service nova] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:26.446869 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02479f37-ea9a-48a8-b944-db7978f410fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:26.448290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02479f37-ea9a-48a8-b944-db7978f410fc service placement] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:26.448290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02479f37-ea9a-48a8-b944-db7978f410fc service placement] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:26.448572 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02479f37-ea9a-48a8-b944-db7978f410fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:26.450818 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-02479f37-ea9a-48a8-b944-db7978f410fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:26.466982 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02479f37-ea9a-48a8-b944-db7978f410fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:26.470086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02479f37-ea9a-48a8-b944-db7978f410fc service placement] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:26.481052 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1454/2908] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:30.001921 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd92cb-101c-4f59-b20a-a9deae48b4b0 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:30.003591 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3bdd92cb-101c-4f59-b20a-a9deae48b4b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:30.017360 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd92cb-101c-4f59-b20a-a9deae48b4b0 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:30.025135 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:30.026247 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:30.026969 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:30.027668 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] PATH_INFO: `/v3/users/893e06c5ea99497fae2847399b7a1a3c` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:30.029825 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=893e06c5ea99497fae2847399b7a1a3c)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:30.035044 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:30.110834 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893e06c5ea99497fae2847399b7a1a3c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.120473 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893e06c5ea99497fae2847399b7a1a3c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.121216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893e06c5ea99497fae2847399b7a1a3c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.138124 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893e06c5ea99497fae2847399b7a1a3c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.138930 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893e06c5ea99497fae2847399b7a1a3c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.168643 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1455/2909] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:30 2022] DELETE /identity/v3/users/893e06c5ea99497fae2847399b7a1a3c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:30.447371 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02479f37-ea9a-48a8-b944-db7978f410fc service placement] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:30.454912 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:30.456454 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:30.461039 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02479f37-ea9a-48a8-b944-db7978f410fc service placement] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:30.524261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1328b6-b240-4fba-a80e-85bbef3a9b06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:30.525191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1328b6-b240-4fba-a80e-85bbef3a9b06 service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:30.525600 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1328b6-b240-4fba-a80e-85bbef3a9b06 service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:30.525974 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1328b6-b240-4fba-a80e-85bbef3a9b06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:30.528382 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4d1328b6-b240-4fba-a80e-85bbef3a9b06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:30.559753 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0c5d5-709c-42f3-8a78-cd9c1020e6b7 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:30.564791 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:30.565456 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:30.565733 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:30.566159 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] PATH_INFO: `/v3/projects/9f1c2adbca494a53a24033b674c2762c` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:30.567472 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1c2adbca494a53a24033b674c2762c)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:30.569802 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:30.578684 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:30.608624 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1328b6-b240-4fba-a80e-85bbef3a9b06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:30.610207 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1328b6-b240-4fba-a80e-85bbef3a9b06 service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:30.622265 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1455/2910] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:59:30.628384 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1c2adbca494a53a24033b674c2762c', 'request_id': 'req-99fc691c-123d-4952-a5cf-caacfcb3eeab'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.635506 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1c2adbca494a53a24033b674c2762c', 'request_id': 'req-99fc691c-123d-4952-a5cf-caacfcb3eeab'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.642192 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1c2adbca494a53a24033b674c2762c', 'request_id': 'req-99fc691c-123d-4952-a5cf-caacfcb3eeab'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.665351 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] The token cache is being invalidate because project 9f1c2adbca494a53a24033b674c2762c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:30.665680 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.666439 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.668826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1456/2911] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:30 2022] DELETE /identity/v3/projects/9f1c2adbca494a53a24033b674c2762c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:59:30.673304 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1328b6-b240-4fba-a80e-85bbef3a9b06 service neutron] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:30.685906 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1328b6-b240-4fba-a80e-85bbef3a9b06 service neutron] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:30.755509 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:30.756934 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:30.757211 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:30.757480 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] PATH_INFO: `/v3/users/f030fc172a6748a2865277207656845b` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:30.758858 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f030fc172a6748a2865277207656845b)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:30.760879 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:30.805653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:30.813125 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f030fc172a6748a2865277207656845b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.826463 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f030fc172a6748a2865277207656845b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.827229 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f030fc172a6748a2865277207656845b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.827857 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f030fc172a6748a2865277207656845b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.836592 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f030fc172a6748a2865277207656845b'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.850711 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1456/2912] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:30 2022] DELETE /identity/v3/users/f030fc172a6748a2865277207656845b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:31.167407 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:31.169517 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:31.183243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc691c-123d-4952-a5cf-caacfcb3eeab admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:31.249243 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:31.249911 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.250187 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.250450 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] PATH_INFO: `/v3/projects/7ad3ee1d713e4186a2d740ffbb154dc0` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.251606 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ad3ee1d713e4186a2d740ffbb154dc0)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:31.253580 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:31.314894 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ad3ee1d713e4186a2d740ffbb154dc0', 'request_id': 'req-c690eccc-5f7a-43e9-8b71-9167364442af'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:31.322589 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ad3ee1d713e4186a2d740ffbb154dc0', 'request_id': 'req-c690eccc-5f7a-43e9-8b71-9167364442af'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:31.329043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ad3ee1d713e4186a2d740ffbb154dc0', 'request_id': 'req-c690eccc-5f7a-43e9-8b71-9167364442af'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:31.353131 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] The token cache is being invalidate because project 7ad3ee1d713e4186a2d740ffbb154dc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:31.353826 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:31.354986 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:31.357721 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1457/2913] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:31 2022] DELETE /identity/v3/projects/7ad3ee1d713e4186a2d740ffbb154dc0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:34.345567 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:34.347413 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:34.457719 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3c84b-db22-46c2-93b0-12fe7746b798 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:34.530146 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3a664-31f5-4e5d-bb21-f4d660d3cbe3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:34.531014 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3a664-31f5-4e5d-bb21-f4d660d3cbe3 service nova] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:34.535272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3a664-31f5-4e5d-bb21-f4d660d3cbe3 service nova] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:34.535272 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3a664-31f5-4e5d-bb21-f4d660d3cbe3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:34.537041 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-7dc3a664-31f5-4e5d-bb21-f4d660d3cbe3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:34.554426 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3a664-31f5-4e5d-bb21-f4d660d3cbe3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:34.556244 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3a664-31f5-4e5d-bb21-f4d660d3cbe3 service nova] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:34.567706 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1457/2914] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:41.347695 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:44.562667 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=5,keystone:SELECT=76,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:37.717144 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:37.718774 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:37.779805 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690eccc-5f7a-43e9-8b71-9167364442af admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:37.783698 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da095fbc-4001-43cc-99c7-a1d270efc222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:37.784278 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da095fbc-4001-43cc-99c7-a1d270efc222 service neutron] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:37.784577 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da095fbc-4001-43cc-99c7-a1d270efc222 service neutron] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:37.784842 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da095fbc-4001-43cc-99c7-a1d270efc222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:37.786585 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-da095fbc-4001-43cc-99c7-a1d270efc222 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:37.798286 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da095fbc-4001-43cc-99c7-a1d270efc222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:37.799750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da095fbc-4001-43cc-99c7-a1d270efc222 service neutron] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:37.808494 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1458/2915] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:37.903553 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3a664-31f5-4e5d-bb21-f4d660d3cbe3 service nova] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:37.913376 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3a664-31f5-4e5d-bb21-f4d660d3cbe3 service nova] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:37.917518 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38278b8-9068-421c-b6d0-983fdf2d484c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff1177119cb48e0a88b0f11a6b1b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:37.918194 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38278b8-9068-421c-b6d0-983fdf2d484c service neutron] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:37.918387 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38278b8-9068-421c-b6d0-983fdf2d484c service neutron] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:37.918663 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38278b8-9068-421c-b6d0-983fdf2d484c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:37.920640 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-d38278b8-9068-421c-b6d0-983fdf2d484c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:37.936928 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38278b8-9068-421c-b6d0-983fdf2d484c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:37.938284 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38278b8-9068-421c-b6d0-983fdf2d484c service neutron] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:37.950290 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1458/2916] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:00:40.068661 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da095fbc-4001-43cc-99c7-a1d270efc222 service neutron] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:40.077689 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-da095fbc-4001-43cc-99c7-a1d270efc222 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:40.109737 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da095fbc-4001-43cc-99c7-a1d270efc222 service neutron] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:40.122194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c319c-5a9e-4836-ba6d-77782dcc7c02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef72bb95992c4a21ae2dbb2a08c51421', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:40.122835 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c319c-5a9e-4836-ba6d-77782dcc7c02 service nova] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:40.123111 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c319c-5a9e-4836-ba6d-77782dcc7c02 service nova] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:40.123420 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c319c-5a9e-4836-ba6d-77782dcc7c02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:40.125216 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a56c319c-5a9e-4836-ba6d-77782dcc7c02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:40.140380 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c319c-5a9e-4836-ba6d-77782dcc7c02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:40.142343 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c319c-5a9e-4836-ba6d-77782dcc7c02 service nova] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:40.156327 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1459/2917] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:00:47.944643 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=6 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:50.149311 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=32 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:53.464159 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec56380-6f0b-4c72-9f6d-d706c822089b None None] REQUEST_METHOD: `POST` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:53.464655 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec56380-6f0b-4c72-9f6d-d706c822089b None None] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:53.464773 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec56380-6f0b-4c72-9f6d-d706c822089b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:53.483470 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.auth.core [None req-4ec56380-6f0b-4c72-9f6d-d706c822089b None None] MFA Rules not processed for user `0a2e5c1014f1441aa47d5de46462b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:53.484672 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4ec56380-6f0b-4c72-9f6d-d706c822089b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:53.538764 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1459/2918] 10.222.0.36 () {58 vars in 967 bytes} [Wed Aug 31 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:53.579176 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c319c-5a9e-4836-ba6d-77782dcc7c02 service nova] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:53.580595 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-a56c319c-5a9e-4836-ba6d-77782dcc7c02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:53.662483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c319c-5a9e-4836-ba6d-77782dcc7c02 service nova] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:53.668222 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3c3c9-ef14-4df5-8f9c-c45221e5f381 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5cf86ae159749cea0cc9783bffbfe75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:53.668829 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c3c9-ef14-4df5-8f9c-c45221e5f381 service swift] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:53.669105 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c3c9-ef14-4df5-8f9c-c45221e5f381 service swift] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:53.669373 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c3c9-ef14-4df5-8f9c-c45221e5f381 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:53.677732 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3c3c9-ef14-4df5-8f9c-c45221e5f381 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:53.679341 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3c3c9-ef14-4df5-8f9c-c45221e5f381 service swift] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:53.687942 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1460/2919] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 10:00:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:00:57.319150 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec56380-6f0b-4c72-9f6d-d706c822089b None None] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:57.320702 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-4ec56380-6f0b-4c72-9f6d-d706c822089b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:57.416411 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec56380-6f0b-4c72-9f6d-d706c822089b None None] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:57.421733 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce36c6-1747-4924-a1c6-10e7bbeceffd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:57.422435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce36c6-1747-4924-a1c6-10e7bbeceffd service placement] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:57.422788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce36c6-1747-4924-a1c6-10e7bbeceffd service placement] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:57.423145 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce36c6-1747-4924-a1c6-10e7bbeceffd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:57.425011 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-00ce36c6-1747-4924-a1c6-10e7bbeceffd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:57.439086 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce36c6-1747-4924-a1c6-10e7bbeceffd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:57.440534 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce36c6-1747-4924-a1c6-10e7bbeceffd service placement] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:57.451966 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1460/2920] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:01:00.560338 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3c3c9-ef14-4df5-8f9c-c45221e5f381 service swift] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:01:00.561889 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-9ef3c3c9-ef14-4df5-8f9c-c45221e5f381 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:00.629120 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3c3c9-ef14-4df5-8f9c-c45221e5f381 service swift] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:01:00.633614 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:01:00.634230 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:01:00.635080 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:01:00.635794 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] PATH_INFO: `/v3/users/f6393be1efdd4c12bcf3adbb230c4943` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:01:00.636597 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6393be1efdd4c12bcf3adbb230c4943)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:01:00.638405 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:01:00.683234 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6393be1efdd4c12bcf3adbb230c4943'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:00.690316 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6393be1efdd4c12bcf3adbb230c4943'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:00.690939 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6393be1efdd4c12bcf3adbb230c4943'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:00.703519 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6393be1efdd4c12bcf3adbb230c4943'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:00.704136 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6393be1efdd4c12bcf3adbb230c4943'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:00.711890 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:01:00.717602 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1461/2921] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 10:01:00 2022] DELETE /identity/v3/users/f6393be1efdd4c12bcf3adbb230c4943 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:01:00.898346 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce36c6-1747-4924-a1c6-10e7bbeceffd service placement] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:01:00.912442 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce36c6-1747-4924-a1c6-10e7bbeceffd service placement] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:01:00.973113 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:01:00.973657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:01:00.973930 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:01:00.974200 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] PATH_INFO: `/v3/projects/3ad415e35f8746ff86bd0d4f96c3ca9e` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:01:00.975362 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ad415e35f8746ff86bd0d4f96c3ca9e)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:01:00.977678 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:01:01.017763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:01:01.030042 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ad415e35f8746ff86bd0d4f96c3ca9e', 'request_id': 'req-12e47305-2c7f-410d-8d69-d3d5d92b2a07'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.039584 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ad415e35f8746ff86bd0d4f96c3ca9e', 'request_id': 'req-12e47305-2c7f-410d-8d69-d3d5d92b2a07'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.046275 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ad415e35f8746ff86bd0d4f96c3ca9e', 'request_id': 'req-12e47305-2c7f-410d-8d69-d3d5d92b2a07'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.058125 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] The token cache is being invalidate because project 3ad415e35f8746ff86bd0d4f96c3ca9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:01:01.058414 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.059095 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.060651 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1461/2922] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 10:01:00 2022] DELETE /identity/v3/projects/3ad415e35f8746ff86bd0d4f96c3ca9e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:01:01.064750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:01:01.066142 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:01.079998 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347316e-e778-4ff7-ae85-3ccb9b54a612 admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:01:01.137788 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:01:01.138410 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:01:01.138688 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:01:01.138958 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] PATH_INFO: `/v3/users/d00fe4ef34724b2bbe1870c82f4ffb3f` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:01:01.140116 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d00fe4ef34724b2bbe1870c82f4ffb3f)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:01:01.142047 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:01:01.189769 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd00fe4ef34724b2bbe1870c82f4ffb3f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.197001 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd00fe4ef34724b2bbe1870c82f4ffb3f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.197714 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd00fe4ef34724b2bbe1870c82f4ffb3f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.209423 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd00fe4ef34724b2bbe1870c82f4ffb3f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.210048 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd00fe4ef34724b2bbe1870c82f4ffb3f'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.222750 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1462/2923] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 10:01:01 2022] DELETE /identity/v3/users/d00fe4ef34724b2bbe1870c82f4ffb3f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:01:01.403448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:01:01.404897 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:01.419653 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e47305-2c7f-410d-8d69-d3d5d92b2a07 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:01:01.480709 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:01:01.481234 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:01:01.481512 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:01:01.481780 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] PATH_INFO: `/v3/projects/9101f0890a934a4baee830f365d9eeaf` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:01:01.482903 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9101f0890a934a4baee830f365d9eeaf)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:01:01.484836 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:01:01.531419 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9101f0890a934a4baee830f365d9eeaf', 'request_id': 'req-8f67e6c1-774f-47c1-b110-4ca208206b06'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.539038 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9101f0890a934a4baee830f365d9eeaf', 'request_id': 'req-8f67e6c1-774f-47c1-b110-4ca208206b06'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.545957 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9101f0890a934a4baee830f365d9eeaf', 'request_id': 'req-8f67e6c1-774f-47c1-b110-4ca208206b06'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.557191 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] The token cache is being invalidate because project 9101f0890a934a4baee830f365d9eeaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:01:01.557462 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.558158 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:01:01.559697 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1462/2924] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 10:01:01 2022] DELETE /identity/v3/projects/9101f0890a934a4baee830f365d9eeaf => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:01:11.219266 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=54,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:01:11.553414 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:12.936008 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:12.937511 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:13.008132 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1936c0f-1893-4baa-96d9-5f7e429e311b admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:13.081896 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcc077-6518-4f85-9ab4-3862aead941a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:13.083053 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcc077-6518-4f85-9ab4-3862aead941a service placement] REQUEST_METHOD: `GET` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:13.084043 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcc077-6518-4f85-9ab4-3862aead941a service placement] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:13.084474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcc077-6518-4f85-9ab4-3862aead941a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:13.087547 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-3cfcc077-6518-4f85-9ab4-3862aead941a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:13.103171 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcc077-6518-4f85-9ab4-3862aead941a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:13.104836 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcc077-6518-4f85-9ab4-3862aead941a service placement] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:13.116916 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1463/2925] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 10:02:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:02:20.956435 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:20.958023 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:21.028920 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67e6c1-774f-47c1-b110-4ca208206b06 admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:21.033510 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:21.034139 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:21.034423 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:21.034695 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] PATH_INFO: `/v3/users/f4003700359449afb83a7352ff2b326a` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:21.035890 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4003700359449afb83a7352ff2b326a)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:21.038715 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:21.093673 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4003700359449afb83a7352ff2b326a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.105634 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4003700359449afb83a7352ff2b326a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.106361 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4003700359449afb83a7352ff2b326a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.106964 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4003700359449afb83a7352ff2b326a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.114264 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4003700359449afb83a7352ff2b326a'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.127948 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1463/2926] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 10:02:20 2022] DELETE /identity/v3/users/f4003700359449afb83a7352ff2b326a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:02:21.314232 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcc077-6518-4f85-9ab4-3862aead941a service placement] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:21.325483 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcc077-6518-4f85-9ab4-3862aead941a service placement] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:21.383022 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:21.383676 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:21.383951 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:21.384221 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] PATH_INFO: `/v3/projects/e73cfdbb50144b67a614836f73ed5b3d` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:21.385334 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e73cfdbb50144b67a614836f73ed5b3d)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:21.387269 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:21.425768 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:21.436537 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e73cfdbb50144b67a614836f73ed5b3d', 'request_id': 'req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.442512 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e73cfdbb50144b67a614836f73ed5b3d', 'request_id': 'req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.448169 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e73cfdbb50144b67a614836f73ed5b3d', 'request_id': 'req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.460908 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] The token cache is being invalidate because project e73cfdbb50144b67a614836f73ed5b3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:02:21.461194 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.461816 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.463318 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1464/2927] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 10:02:21 2022] DELETE /identity/v3/projects/e73cfdbb50144b67a614836f73ed5b3d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:02:21.467456 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:21.468875 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:21.482261 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ab6b9-b19b-4370-bc98-244f0656b90c admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:21.543325 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:21.543980 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:21.544257 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:21.544529 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] PATH_INFO: `/v3/users/51abe0e78eef423ca0bd6b80533311bb` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:21.545703 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=51abe0e78eef423ca0bd6b80533311bb)` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:21.547788 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:21.573448 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:21.606654 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51abe0e78eef423ca0bd6b80533311bb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.620657 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51abe0e78eef423ca0bd6b80533311bb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.621595 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51abe0e78eef423ca0bd6b80533311bb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.622251 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51abe0e78eef423ca0bd6b80533311bb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.631491 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.notifications [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51abe0e78eef423ca0bd6b80533311bb'} {{(pid=93532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.644763 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1464/2928] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 10:02:21 2022] DELETE /identity/v3/users/51abe0e78eef423ca0bd6b80533311bb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:02:21.821474 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] Authenticating user token {{(pid=93533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:21.823566 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.fernet_utils [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:21.855631 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37cc9e-e97a-49d5-98d0-b36ffe439c5a admin admin] Validating token access rules against request {{(pid=93533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:21.916295 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670cae4-62f4-462e-9579-cce34d53926c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7063e9478d747e983e7ced895545d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd473eacf42fc47c2aa82e80873c7637e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:21.916925 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670cae4-62f4-462e-9579-cce34d53926c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:21.917200 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670cae4-62f4-462e-9579-cce34d53926c admin admin] SCRIPT_NAME: `/identity` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:21.917466 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670cae4-62f4-462e-9579-cce34d53926c admin admin] PATH_INFO: `/v3/projects/ef85975f274544618475ececc91cda31` {{(pid=93533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:21.918577 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670cae4-62f4-462e-9579-cce34d53926c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef85975f274544618475ececc91cda31)` {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:21.920509 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670cae4-62f4-462e-9579-cce34d53926c admin admin] RBAC: Authorization granted {{(pid=93533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:21.970319 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d670cae4-62f4-462e-9579-cce34d53926c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef85975f274544618475ececc91cda31', 'request_id': 'req-d670cae4-62f4-462e-9579-cce34d53926c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.976443 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d670cae4-62f4-462e-9579-cce34d53926c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef85975f274544618475ececc91cda31', 'request_id': 'req-d670cae4-62f4-462e-9579-cce34d53926c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.982125 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d670cae4-62f4-462e-9579-cce34d53926c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef85975f274544618475ececc91cda31', 'request_id': 'req-d670cae4-62f4-462e-9579-cce34d53926c'} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.996045 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d670cae4-62f4-462e-9579-cce34d53926c admin admin] The token cache is being invalidate because project ef85975f274544618475ececc91cda31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:02:21.996325 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d670cae4-62f4-462e-9579-cce34d53926c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.996962 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG keystone.notifications [None req-d670cae4-62f4-462e-9579-cce34d53926c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:21.998630 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: [pid: 93533|app: 0|req: 1465/2929] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 10:02:21 2022] DELETE /identity/v3/projects/ef85975f274544618475ececc91cda31 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:02:31.641120 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG dbcounter [-] [93532] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=93532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:31.992914 dv-cinder-837543-8-iscsi devstack@keystone.service[93533]: DEBUG dbcounter [-] [93533] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=93533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:07:41.199882 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] Authenticating user token {{(pid=93532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:41.201337 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:41.343916 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae5dd2-ae23-4eb6-aeda-3df05fed2f3b admin admin] Validating token access rules against request {{(pid=93532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:41.404593 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe5bd2-a4b9-4703-8282-21f2c619b42a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '560df3eb05f14b6489158ab053f282ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef8b586382941408cd88ee654c42d32', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:41.405205 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe5bd2-a4b9-4703-8282-21f2c619b42a service placement] REQUEST_METHOD: `GET` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:41.405483 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe5bd2-a4b9-4703-8282-21f2c619b42a service placement] SCRIPT_NAME: `/identity` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:41.405754 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe5bd2-a4b9-4703-8282-21f2c619b42a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:41.407552 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.fernet_utils [None req-6ebe5bd2-a4b9-4703-8282-21f2c619b42a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:41.419132 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe5bd2-a4b9-4703-8282-21f2c619b42a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:41.420725 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe5bd2-a4b9-4703-8282-21f2c619b42a service placement] RBAC: Authorization granted {{(pid=93532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:41.431500 dv-cinder-837543-8-iscsi devstack@keystone.service[93532]: [pid: 93532|app: 0|req: 1465/2930] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 10:07:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0)